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Preface 


For  the  twenty-fifth  year,  the  Research  and  Theory  Division  of  the  Association  for  Educational  Communications  and 
Technology  (AECT)  is  sponsoring  the  publication  of  these  Proceedings.  Papers  published  in  this  volume  were 
presented  at  the  National  AECT  Convention  in  Dallas,  TX.  A limited  quantity  of  these  Proceedings  were  printed 
and  sold  in  both  hardcopy  and  electronic  versions.  It  is  also  available  on  microfiche  through  the  Educational 
Resources  Clearinghouse  (ERIC)  System. 

The  Proceedings  of  AECT’s  Convention  are  published  in  two  volumes.  Volume  #1  contains  papers  dealing 
primarily  with  research  and  development  topics.  Papers  dealing  with  instruction  and  training  issues  are  contained  in 
Volume  #2.  Volume  #2  contains  over  60  papers. 

REEEREEING  PROCESS:  Papers  selected  for  presentation  at  the  AECT  Convention  and  included  in  these 
Proceedings  were  subjected  to  a reviewing  process.  All  references  to  authorship  were  removed  from  proposals 
before  they  were  submitted  to  referees  for  review.  Approximately  fifty  percent  of  the  manuscripts  submitted  for 
consideration  were  selected  for  presentation  at  the  convention  and  for  publication  in  these  Proceedings,  the  papers 
contained  in  this  document  represent  some  of  the  most  current  thinking  in  educational  communications  and 
technology. 
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First  Online  Collaborative  Learning  Experiences  of  Freshmen  Students  in 

Turkey 


Cengiz  Hakan  Aydin 
Anadolu  University 
School  of  Communication  Scierces 
Eskisehir,  Turkey 


Abstract 

In  literature,  there  is  very  few  significant  study  on  the  effectiveness  of  online  collaborative  learning  on 
students  who  had  no  prior  experience  of  collaboration.  This  presentation  describes  a case  study  in  which  attitudes 
of  students  who  had  no  prior  experience  in  online  learning  and  collaborative  learning  toward  online  collaborative 
learning  in  Turkey  were  investigated.  Data  have  been  collected  by  using  a Likert  type  questionnaire  and  interviews. 
Students  mostly  expressed  positive  attitudes  toward  online  learning  while  they  indicated  negative  attitudes  toward 
collaborative  learning.  These  negative  attitudes  associated  with  the  preconceptions  of  learners  about  collaboration 
and  not  enough  prior  experience  to  develop  interpersonal  skills.  However,  majority  of  the  students  pointed  out  that 
they  would  like  to  take  similar  courses  online  and  study  collaboratively. 

Introduction 

The  field  of  education  has  witnessed  impressive  changes  during  the  last  decade  with  the  developments  of 
new  communication  technologies.  These  technologies  helped  faculties  provide  more  interactive  learning  and  apply 
new  pedagogical  approaches  in  their  courses. 

One  of  these  approaches  that  the  communication  technologies,  especially  Internet  enabled  is  collaborative 
learning.  Literature  has  revealed  that  collaborative  learning  in  classroom  settings  offer  many  benefits  such  as  active 
learning  and  peer  interaction  for  learners.  Many  of  these  studies  were  carefully  controlled  to  test  the  hypothesis  that 
collaborative  learning  was  more  effective  then  other  modes  of  instructional  delivery.  The  evidence  for  the 
effectiveness  of  peer  teaching  is  well  accepted  for  a wide  range  of  goals,  content,  and  students.  This  effectiveness 
has  been  repeatedly  documented  for  the  past  90  years.  Klemm  (1994)  reports  that  over  575  experimental  and  100 
correlation  studies  have  been  conducted  by  a wide  variety  of  researchers  who  tested  subjects  of  differing  age, 
cultures,  and  geographical  areas.  Most  of  these  studies  have  revealed  that  collaborative  learning  is  more  effective 
than  most  other  approaches.  The  reasons  of  this  effectiveness  summarized  by  Smith  (1993)  as:  (1)  the  student  who 
learns  best  is  the  one  who  organizes,  summarizes,  elaborates,  explains,  and  defends;  (2)  more  learning  occurs  in  an 
environment  of  peer  support  and  encouragement  because  students  eagerly  work  harder  and  longer;  (3)  students  learn 
more  when  they're  doing  things  they  enjoy. 

Developing  body  of  research  on  online  collaborative  learning  has  also  shown  that  students  get  benefits 
from  distance  and/or  online  collaboration.  The  researchers  and  writers  such  as  Mason  and  Kaye  (1990),  Harasim 
(1990),  Henri  and  Rigault  (1996)  have  described  the  potential  of  the  Internet  as  an  interactive  environment  that 
would  enable  collaborative  group  learning  and  would  change  the  nature  of  distance  education  from  an  autonomous, 
isolated  experience  to  a potentially  social  constructivist  environment. 

In  the  years  that  the  Internet  has  come  into  more  widespread  use  for  tertiary  learning,  online  collaborative 
learning  has  become  more  commonly  accepted  as  an  effective  strategy  that  is  now  made  possible  by  the  technology. 
Although  literature  has  shown  that  online  collaborative  learning  has  become  a hot  topic  for  researchers  and 
practitioners  all  over  the  world,  there  is  almost  no  or  a few  significant  study  on  the  effectiveness  of  online 
collaborative  learning  on  students  who  had  no  prior  experience  of  collaboration,  especially  in  Turkey. 

Collaborative  Learning  in  Turkey 

In  Turkey,  most  of  the  instruction  almost  all  levels  of  education  except  higher  education  have  a very 
instructive  structure.  Teachers  use  textbooks,  which  are  recommended  by  the  Ministry  of  Education  and  prepared 
according  to  the  curriculum,  during  the  instruction.  They  mostly  present  the  instruction  orally  and  conduct  drill  and 
practice  sessions.  Sometimes  enthusiastic  teachers  provide  opportunity  for  students  to  do  hands  on  activities, 
discussions,  inquiries.  Although  the  Ministry  of  Education  has  recently  started  to  encourage  teachers  to  use 


collaborative  activities  in  their  classes,  almost  no  teachers  use  this  approach  due  to  lack  of  skills  and  time.  So  the 
Turkish  higher  education  students  are  not  used  to  work  in  teams. 

In  addition,  online  education  is  not  common  in  primary  and  secondary  education  in  Turkey.  So  that  almost 
all  Turkish  students  come  to  universities  with  no  online  learning  experience.  Even  worst,  almost  half  of  them  enter 
universities  with  a few  computer  literacy  skills. 

Purpose  and  Method  of  the  Study 

This  paper  describes  a study  in  which  attitudes  of  undergraduate  students  who  had  had  no  online  learning 
and  collaborative  learning  experience  before  having  taken  “Introduction  to  Educational  Communications’’  cours  e 
online  in  School  of  Communication  Sciences  Anadolu  University  in  Turkey  are  investigated.  In  other  words,  the 
main  focus  of  this  study  was  to  explore  the  attitudes  of  the  students  toward  online  collaborative  education. 

“Introduction  to  Educational  Communications’’  is  a 14  weeks  long  compulsory  course  for  the  freshmen 
students  in  the  Educational  Communications  and  Planning  Department.  After  three  hours  workshop  on  online 
learning,  collaborative  learning  and  online  collaborative  learning,  the  students  were  asked  to  participate  voluntarily 
the  study  and  take  the  course  online.  Only  19  out  of  45  students,  who  enrolled  this  course,  participated  to  the  study 
as  the  subject  group. 

After  two  weeks  of  preliminary  preparations,  subjects  took  the  course  entirely  online  during  the  spring 
2001  academic  semester.  WebCT  was  used  as  the  online  learning  and  management  environment.  At  the  end  of  the 
course  they  were  asked  to  fill  the  questionnaire  and  attend  the  interviews.  Eor  the  data  collection,  a thirty  items 
Likert  type  questionnaire  was  used.  Eirst  ten  items  in  the  questionnaire  were  developed  to  learn  the  students’ 
reaction  on  online  learning  and  the  last  ten  items  were  related  to  the  collaborative  learning.  Also,  interviews  were 
conducted  with  the  four  of  the  randomly  selected  students  to  get  a better  in -sight  about  the  students’  responses.  The 
participants’  final  grades  are  used  for  measuring  their  achievements. 

Results 

The  study  sought  to  explore  the  responses  of  the  learners  to  online  learning  environment  and  collaborating 
in  this  environment.  The  questionnaire  at  the  end  of  the  course  asked  students  to  provide  their  responses  to  a series 
of  questions  on  these  topics.  He  questionnaire  used  a Likert  scale  responses  with  five  categories  arranged  so  that  the 
3.41  mean  score  indicated  an  expected  level  of  agreement  with  the  item  while  other  responses  enables  students  to 
show  higher  or  lower  levels  of  agreement.  The  3.41  mean  average  was  determined  after  identifying  the  critical  level: 
4 intervals  / 5 categories  = 0.8 . As  a result  of  this  the  levels  of  agreement  were  determined  as  followings: 

1 (Strongly  Disagree)  = l-i-  0.8  =1.8 

2 (Disagree)  = 1.8 -i- 0.8  = 2.6 

3 (Not  sure)  = 2.6  -i-  0.8  = 3.4 

4 (Agree)  = 3.4 -i- 0.8  = 4.2 

5 (Strongly  Agree)  =4.2 -1-0.8  = 5 

In  interpreting  the  results,  3.41  mean  average  was  taken  as  what  would  be  expected  from  agreed  students  and  higher 
scales  (agree  and  strongly  agree)  and  the  lower  scales  (disagree  and  strongly  disagree). 

Online  learning.  Student’s  responses  to  questions  concerning  their  impressions  of  the  online  learning  were 
generally  positive  (M=3,768).  However,  for  the  9^^  item  which  is  “I  can  remember  better  what  I learn  when  I study 
online’’,  average  mean  of  students’  responses  is  lower  then  critical  level  of  agreement  (M9=3,158>M=3.41).  Also, 
students  scored  higher  on  the  item  #1  (It  was  interesting  to  study  the  topic  online),  item  #3  (I  liked  working  online) 
and  item  #4  (I  found  easier  to  study  online). 

Table  1:  Frequency  of  the  students’  responses  to  the  items  related  to  online  learning 


Item 

# 

I (Strongly 
Disagree) 

2 

(Disagree) 

3 

(Not  Sure) 

4 

(Agree) 

5 (Strongly 
Agree) 

M 

I 

0 

2 

1 

11 

5 

4,000 

2 

0 

1 

5 

11 

2 

3,737 

3 

0 

1 

5 

6 

7 

4,000 

4 

0 

2 

3 

7 

7 

4,000 

5 

0 

3 

4 

7 

5 

3,737 

6 

0 

3 

6 

5 

5 

3,632 

2 


7 


4 


0 1 
8 0 2 

9 0 7 

10  0 2 


9 5 3,947 

5 4 3,579 

6 2 3,158 

_5 7 3,895 

means  score  average  3, 768 


Collaborative  learning.  The  collaborative  learning  activities  tended  to  be  viewed  quite  negatively  by  the  majority 
of  the  students  (M=2,868).  By  looking  at  students’  scores,  it  can  be  told  that  they  were  having  problems  with  their 
teammates  all  the  time  (M 25=2,789)  and  were  feeling  discomfort  working  in  the  team  (M26=2,684).  Most  of  the 
students  also  expressed  that  they  would  prefer  working  individually  to  in  teams. 


Table  2:  Frequency  of  the  students’  responses  to  the  items  related  to  collaborative  learning 


Item 

# 

I (Strongly 
Disagree) 

2 

(Disagree) 

3 

(Not  Sure) 

4 

(Agree) 

5 (Strongly 
Agree) 

M 

21 

2 

4 

6 

5 

2 

3,053 

22 

1 

7 

7 

3 

1 

2,789 

23 

1 

6 

7 

4 

1 

2,895 

24 

1 

6 

6 

6 

0 

2,895 

25 

2 

8 

4 

5 

0 

2,632 

26 

1 

9 

4 

5 

0 

2,684 

27 

2 

5 

4 

8 

0 

2,947 

28 

2 

3 

9 

4 

1 

2,947 

29 

1 

3 

7 

7 

1 

3,211 

30 

7 

3 

3 

2 

4 

2,684 

means  score  average 

2,868 

At  the  end  of  the  questionnaire,  students  were  asked  how  they  would  like  to  take  a similar  course  in  the  future  and 
four  different  instructional  delivery  formats  offered  the  students.  These  were  (1)  online  and  collaborative,  (2)  online 
and  individualized,  (3)  face-to-face  and  collaborative,  (4)  face-to-face  and  individualized.  Table  3 shows  the  results 
of  this  question.  Most  of  the  students  (69.42%)  chosen  to  take  similar  courses  online  but  interestingly  majority  of 
them  (36.84%)  also  favored  online  collaborative  learning.  In  addition,  some  of  the  students  who  indicated  that  they 
would  not  like  to  take  a course  online  prefer  collaborative  learning.  With  these  students  total  percentage  of  the 
students  who  favored  collaborative  learning  is  52.63.  Furthermore,  only  31.58  % of  the  students  wanted  to  take 
similar  courses  face-to-face. 

Table  3:  Students’  preferences  of  delivery  formats  for  the  future  courses 


Delivery 

Formats 

Number  of  the  Students 
(%) 

online  and  collaborative 

7 (36.84) 

online  and  individualized 

6 (32.58) 

face-to-face  and  collaborative 

3(15.79) 

face-to-face  and  individualized 

3(15.79) 

TOTAL 

19(100) 

Interviews.  During  the  interviews,  the  students  (randomly  selected  four  students)  expressed  several  common 
thoughts  about  the  online  learning  and  collaborative  learning.  One  of  these  common  thoughts  was  about  their 
distress  with  the  technology.  The  server  was  down  due  to  some  technical  problems  during  two  weeks.  So  neither 
instructor  not  students  was  able  to  reach  the  web  site  and  email  messages.  This  unpleasant  situation  created  a 
discomfort  among  students  toward  online  learning.  Another  thought  about  online  learning  is  that  most  of  the 
students  took  the  course  online  because  they  heart  a lot  about  online  education  but  did  not  know  anything.  So  they 
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thought  this  could  be  an  opportunity  for  them  to  learn  this  new  environment.  However,  after  the  course,  they  found 
that  since  they  were  coming  to  school  for  other  courses  it  was  not  worth  to  take  the  course  online.  Students  were 
also  indicated  that  it  was  hard  to  find  a computer  to  use  especially  during  the  mid-term  and  finals.  So,  they  believe 
that  online  learners  should  have  computers  at  home  to  be  successful. 

On  the  other  hand,  students  felt  that  because  they  did  not  know  classmates  well,  they  did  not  work  together 
efficiently.  They  indicated  that  it  took  several  weeks  to  understand  who  in  the  team  was  working  hard  and  who 
needed  more  encouragement  and  push.  They  complained  about  the  students  who  had  chosen  to  do  nothing  but  using 
their  teammates.  Besides,  they  claimed  that  if  they  had  chance  to  work  with  their  close  friends  in  teams,  the  team 
work  could  have  been  better. 

Discussions  and  Conclusions 

The  study  has  revealed  several  findings,  most  of  which  agreed  with  the  literature.  Findings  related  to  the 
students’  distress  with  this  online  course  due  to  technological  problems  are  similar  to  the  ones  that  Hara  and  Kling 
(2000),  for  instance,  have  reported.  Online  learning  is  a hot  topic  in  Turkey  but  there  are  few  attempts  to  offer  online 
courses  and  programs.  One  of  the  significant  reasons  of  this  shortage  is  the  technological  problems  such  as  low 
bandwidth,  low  home  computer  ownership,  and  lack  of  computer  literacy  skills  of  both  instructors  and  learners 
(Aydin,  2001).  Due  to  these  problems  happened  during  the  study,  the  students  did  not  find  what  they  were  expecting 
before  the  course  about  online  learning.  However,  they  still  showed  a higher  level  of  positive  attitude  toward  online 
learning. 

If  online  learning  advocates  and  providers  in  Turkey  do  not  want  to  loose  their  customers,  they  should  offer 
better  examples  of  online  learning.  Especially  for  the  learners  who  are  able  to  come  to  campus,  providers  should 
design  the  courses  in  a way  that  these  students  can  get  benefit  of  them  and  value  the  potential  of  online  learning.  For 
example  using  a blended  mode  to  create  constructivist  learning  environments  or  online  learning  communities  among 
on-campus  students  might  be  a way  of  uncovering  the  benefits  of  online  learning.  Providers  should  not  ignore  that 
today’s  on-campus  students  might  not  be  able  to  come  to  campuses  to  improve  their  skills  or  to  get  a higher  level  of 
education  (e.g.  masters’  degree  or  PhD)  after  the  school.  So  it  is  important  to  find  the  right  mode  of  online  learning 
for  the  learners  and  train  them  about  online  learning  before  actually  start  an  online  course. 

On  the  other  hand,  it  can  be  claimed  that  the  Turkish  education  system  is  highly  competitive.  That  is  it 
forces  students  work  alone  and  hard  to  be  able  to  get  in  good  schools  in  every  level,  although  over  the  recent  years 
the  Ministry  of  Education  tries  to  encourage  the  teachers  to  use  collaborative  activities  in  their  classes.  So  that 
students  develop  positive  attitudes  towards  individualized  and  competitive  learning  rather  then  collaborative 
learning.  They  come  to  the  higher  education  with  this  preconception.  The  students  participated  with  these 
preconceptions  to  the  study.  As  it  is  indicated  in  literature,  students  should  be  trained  before  a collaborative  learning 
experience.  That  is  why  the  students  were  trained  about  online  learning  and  collaboration  during  the  first  two  weeks 
of  the  study.  However,  the  study  shows  that  those  two  weeks  were  not  enough  for  them  to  acquire  social  and 
interpersonal  skills  to  complete  a collaborative  activity.  Therefore,  before  starting  an  online  collaborative 
experience,  the  designers  must  be  sure  that  their  students  have  developed  a satisfied  level  of  interpersonal  and  social 
skills  that  will  help  students  while  they  are  collaborating  to  learn  online. 

Besides,  online  collaborative  learning  does  not  require  that  all  the  activities  must  be  online  for  on-campus 
students.  In  other  words,  some  parts  of  collaborative  work  can  be  face-to-face  if  the  students  have  time  to  come  to 
the  school.  For  instance,  discussions  among  team  members  can  be  held  in  face-to-face  meetings  and  presentation  of 
the  results  of  these  discussions  can  be  posted  online.  There  are  many  ways  of  designing  collaborative  activities  and 
it  is  important  to  find  the  right  blend  for  the  learners. 

Overall,  this  study  shows  that  although  students  had  problems  in  collaborating  online  with  their  classmates 
and  the  course  did  not  fulfill  their  expectations  about  online  learning,  they  still  would  like  to  have  more  online 
learning  experiences  and  favor  studying  collaboratively. 

In  the  future  research  studies,  effects  of  computer  literacy,  group  composition,  readiness  for  online  learning 
in  terms  of  computer  ownership,  online  and/or  distance  learning  self-efficacy,  learning  styles  and  strategies,  prior 
experiences  on  students’  achievement  and  attitudes  in  online  collaborative  learning  can  be  investigated.  Various 
comparisons  such  as  face-to-face  versus  online  collaboration,  undergraduate  versus  graduate  students  might  also  be 
conducted  in  the  future. 


4 


References 

Aydin,  C.H.  (2001).  Uses  of  the  Internet  in  Turkey.  Educational  Technology  Research  and  Development 
49(A),  120-123. 

Kara,  N.  & Kling,  R.  (2000).  Students’  distress  with  a web-based  distance  education  course:  An 
ethnographic  study  of  participants’  experiences.  Retrieved  at  1 1.8.2000  at  http://shs.indiana.edu/CSI/wp00-01.html 
Harasim,  L.,  Ed.  (1990).  Online  education:  Perspectives  on  a new  medium.  New  York:  Prager/Greenwood 
Henri,  F.,  & Rigault,  C.  (1996).  Collaborative  distance  education  and  computer  conferencing.  In  T Liao 
(Ed.),  Advanced  educational  technology:  research  issues  and  future  potential  (pp.  45-76).  Berlin:  Springer- Verlag. 

Klemm,  W.  R.  (1994).  Using  a formal  collaborative  learning  paradigm  for  veterinary  medical  education. 
Journal  of  Veterinary  Medical  Education  21, 2-6. 

Mason,  R.,  and  Kaye,  T.  (1990).  Toward  a New  Paradigm  for  Distance  Education.  In  Harasim,  L.  M.  (Ed.), 
Online  Education  Perspectives  on  a New  Environment  (pp.  15-38).  New  York:  Praeger. 

Smith,  B.L.  (1993).  Creating  learning  communities.  Liberal  Education  79(4),  32-39. 


5 


A Comparative  Study  of  a Project-Based  Course:  Face  to  Face  and  Online 


Bonnie  Armstrong 
Hong  Gao 
E Shen 

Florida  State  University 


Abstract 

This  study  was  an  examination  of  the  differences  in  group  communication  patterns  between  two  different 
sections  of  a project-based  course.  One  was  a face-to-face  class  while  the  other  was  an  online  learning  class.  Small 
group  interactions,  learner  attitudes,  and  learning  achievement  are  compared.  Results  show  that  there  was  no 
difference  between  face  to  face  and  online  learning  achievement;  however  face  to  face  and  online  learners  differed 
in  types  of  interactional  statements,  flexibility  in  group  roles,  and  degree  of  learner  comfort  early  in  the  course. 

Introduction 

As  applications  of  computer- mediated  communication  become  more  and  more  accepted  for  teaching, 
educators  will  need  to  understand  the  strengths  and  constraints  of  such  educational  delivery  modes  and  be  able  to 
develop  strategies  that  can  support  communication  and  learning  in  these  environments.  Computer- mediated 
communication  includes  synchronous  or  asynchronous  communication  tools  such  as  electronic  mail,  discussion 
boards,  and  computer  conferencing.  In  this  type  of  communication  senders  encode  the  messages  in  text  that  are 
relayed  from  the  senders'  computers  to  a website  accessed  by  everyone  or  to  one  or  more  selected  receivers' 
computers  (Walther,  1992). 

The  most  common  theories  of  computer  mediated  communication  such  as  social  presence  theory  and  media 
richness  theory  note  that  electronic  mails,  discussion  boards,  and  virtual  chat  eliminate  nonverbal  cues  that  are 
generally  rich  in  the  face-to-face  communication  environment.  The  absence  of  such  codes  affects  the  user’s 
perceptions  of  communication  content  and  constrains  the  users'  interpretations  of  messages.  Such  characteristics 
raise  the  very  real  possibility  that  computer- mediated  communication  may  be  less  suitable  for  certain 
communication  purposes  (Rice,  1984,  see  also  Trevino,  1987)  or  that  there  may  be  negative  effects  on  the 
communication  process  (Jonassen,  & Kwon,  2001;  Hiltz,  Johnson,  & Truroff  1986)  and,  hence,  on  the  learning 
process. 

In  order  to  understand  the  effectiveness  of  the  computer  mediated  system,  therefore,  it  is  essential  to 
understand  the  communication  process  involved  in  interactions  between  instructors  and  students  and  among  students 
in  such  systems.  Perhaps  one  of  the  most  feasible  and  useful  ways  to  determine  the  strength  and  limitations  in 
computer-mediated  communication  is  to  examine  the  similarities  and  differences  of  the  communication  process  in 
computer-mediated  learning  environments  with  the  face-to-face  (F2F)  learning  environment. 

Past  Research 

The  findings  to  date  concerning  the  effects  of  computer-mediated  communication  on  the  learning  process 
have  been  mixed.  Research  has  demonstrated  that  computer-mediated  communication  in  a small  group  teaching- 
learning process  creates  more  flexible  communication  patterns  (Berge  & Collins,  1996;  Heller  & Kearsly,  1996; 
Ruberg,  Moore,  & Taylor,  1996).  Computer  mediated  communication  allows  students  to  interact  with  their  groups 
members  in  a time  that  convenient  for  them  (Berge  & Collins,  1996;  Heller  & Kearsly,  1996;  Ruberg,  Moore,  & 
Taylor,  1996)  and  may  increase  students'  sense  of  responsibility  and  self  discipline  (Berge  & Collins,  1996;  His  & 
Hoadley,  1997). 

Computer  mediated  communication  may  also  affect  the  social  relationships  of  learners  and  the  process  and 
content  of  group  discussions  and  individual  perspectives.  Social  relationships  may  be  equalized  during  interactions 
by  the  lack  of  social  cues  and  cultural  differences  (Berge  & Collins,  1996;  His  & Hoadley,  1997).  Such  changes  in 
relationships  may  also  open  learners  to  new  meanings  and  perspectives  (Heller  & Kersley,  1996;  Ruberg,  Moore,  & 
Taylor,  1996).  The  need  to  articulate  one’s  arguments  in  computer  mediated  communication  in  a small  group  forces 
group  members  to  put  their  thoughts  into  a writing  in  a way  that  others  can  understand  (Valacich  et  al.  1994). 

However,  computer  mediated  communication  can  also  create  difficulties  for  some  students,  especially  in 
small  group  learning  environments.  For  those  students  who  are  in  need  of  structure  and  guidance,  the  increase  in 
responsibility  can  be  problematic  (Berge  & Collins,  1996).  And  the  lack  of  the  social  cues  in  a small  group  may 
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actually  contribute  to  students  becoming  antagonistic  toward  others.  Computer-mediated  communication,  especially 
in  an  asynchronous  mode,  may  disrupt  the  natural  flow  of  conversation  in  a small  group,  removing  the  discourses 
from  its  logical  context  (Kelly  & McQath,  1990).  This  can  make  the  communication  process  in  a small  group  not 
only  inefficient,  but  can  also  lead  to  misinterpretation  of  meaning  and  negative  learner  attitudes  toward  the  learning 
experience. 

The  most  recent  research  in  computer- mediated  group  communication  is  Jonassen  and  Kwon’s  (2001) 
study  comparing  the  communication  patterns  within  the  groups  that  were  collaboratively  solving  problems  via 
computer  conferencing.  Through  the  use  of  cluster  analysis  of  the  communication  patterns,  the  study  concluded  that 
computer-mediated  communication  is  more  task-oriented  and  more  focused  in  a small  group  involved  in  complex 
problem-solving.  This  may  be  the  result  of  the  computer  mediated  communication  system  giving  learners  more  time 
for  critical  thinking  and  composition  of  their  responses.  Another  possibility,  not  mentioned  by  this  study,  is  that  off 
task  comments  in  asynchronous  environments  usually  may  not  generate  responses  from  other  learners  fast  enough  to 
get  such  dialogue  going. 

Despite  the  growth  of  studies  in  computer- mediated  communications  involving  small  groups,  there  is  no 
research  comparing  F2F  and  online  learners  involved  in  intensive  small  group  teams  throughout  a semester  in  a 
project-based  learning  environment.  Nor  have  the  perceptions  and  attitudes  of  such  learners  been  compared 
regarding  their  roles  in  the  group,  feelings  about  the  group  processes  and  preferences  for  communication  tools,  and 
their  evaluation  of  their  own  learning  outcomes  in  such  a learning  environment.  Because  of  this  lack  of  empirical 
evidence  for  small  group  effectiveness  in  the  online  learning  environment,  an  exploratory  study  was  conducted  to 
investigate  questions  related  to  patterns  of  interaction  and  group  work  processes  in  the  online  learning  environment 
using  computer-mediated  communication  tools. 

Research  Questions 

The  purpose  of  this  exploratory  study  was  to  compare  the  similarities  and  differences  in  communication 
patterns  and  related  learner  perceptions  and  attitudes  for  learners  working  in  small  groups  in  F2F  and  online  learning 
environments.  Specifically,  this  study  examined:  1)  project  team  (small  group)  interactions  to  see  if  the  findings  of 
the  Jonasseon  and  Kwon  (2001)  study  would  be  replicated  in  terms  of  more  task-related  statements  for  F2F  learners; 
and  to  determine  what  types  of  interaction  were  most  common  for  each  type  of  learning  environment;  2)  learner 
attitudes  and  perceptions  to  determine  if  there  were  differences  between  F2F  and  online  learners  in  regard  to  their 
feelings  about  collaborative  group  processes  and  their  own  roles,  communication  tools,  and  their  own  learning 
outcomes;  and  3)  learner  achievements  to  see  if  quality  of  products  and  course  grades  were  different  for  F2F  versus 
online  learners. 

The  research  questions  to  be  answered  in  the  current  study  were  as  follows: 

1.  Are  small  groups  working  in  online  courses  more  task- focused  in  their  small  group  communications  than  F2F 
learners? 

2.  What  types  of  interaction  are  most  common  in  the  two  learning  environments? 

3.  Are  online  learners  as  comfortable  in  their  team  roles  as  learners  in  the  F2F  teams? 

4.  Are  online  learners  as  satisfied  with  the  communication  processes  for  group  work  as  F2F  learners? 

5.  Which  communication  tools  are  most  useful  for  online  learners? 

6.  Are  the  learning  outcomes  (achievement  of  course  goals  as  measured  by  team  assignment  grades)  the  same  for 
F2F  and  the  online  learners? 

Method 

Independent  Variable 

In  order  to  compare  interaction  patterns  and  learner  perceptions  and  attitudes,  the  delivery  of  one  course 
during  one  semester  with  two  different  conditions  of  delivery  was  the  independent  variable.  One  section  of  the 
course  was  given  face-to-face  (F2F)  with  weekly  classroom  meetings  and  with  a course  website  used  for  e-mail 
posting  of  information  and  a few  activities  (web-assisted).  The  other  section  of  the  course  was  given  completely 
online  through  the  use  of  a computer- mediated  system.  Students  from  one  section  did  not  interact  or  contact  each 
other,  and  each  viewed  and  used  only  the  materials  presented  to  their  sections  or  available  on  their  own  websites. 
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Dependent  Variables 

To  measure  the  results  of  the  two  different  learning  environments,  the  dependent  variables  selected  were 
the  1)  communication  patterns  of  the  small  groups  working  in  each  type  of  learning  environment;  2)  the  perceptions 
and  attitudes  of  the  learners  concerning  team  processes,  communication  tools,  and  quality  of  assignments;  and  3)  the 
grades  for  course  assignments  (products  each  group  had  to  develop  to  fulfill  assignment  requirements). 

Two  Learning  Conditions 

An  extensive  body  of  literature  supports  the  notion  that  a well -designed  course  will  be  effective  in  the  face- 
to-face  learning  environment;  (Dick  & Carey,  1996;  Gagne,  Briggs,  & Wager,  1988)  and  some  studies  have  now 
included  web-based  learning  environments  as  well  (Bichelmeyer  et  al.,  2001;  Relan  & Gillani,  1997).  Using 
principles  of  good  course  design  and  adaptation  for  the  web,  a graduate  level  education  course.  The  Management  of 
Instructional  Development,  was  designed  and  developed  based  on  an  existing  course  that  is  part  of  the  Instructional 
Systems  master's  program  at  Florida  State  University.  The  new  online  version  of  the  course  could  be  used  to  satisfy 
a management  requirement  in  both  the  existing  face-to-face  graduate  program  as  well  as  for  the  relatively  recently 
implemented  Master's  in  Distance  Learning  also  offered  through  the  Instructional  Systems  Program. 

In  the  F2F  version  the  instructor  and  students  met  for  2 3/4  hours  once  a week  for  16  weeks  with 
attendance  required.  Most  of  the  activities  of  the  F2F  version  class  were  conducted  in  class,  but  the  course  did  have 
a website  available  to  these  learners.  On  the  website  were  slides  used  during  class,  description  of  assignments,  and 
directions  for  two  online  learning  activities  resulting  in  threaded  discussions  for  two  of  the  sixteen  weeks  In 
addition,  all  the  communication  tools  of  small  group  discussion  boards,  virtual  chat,  and  e-mail  were  available  for 
F2F  learners  but  the  instructor  did  not  specify  making  use  of  them,  just  that  they  were  there. 

The  course.  Management  of  Instructional  Development,  is  project-based  and  requires  extensive  group 
collaboration  throughout  the  semester.  The  F2F  version  has  been  given  many  years  and  has  a stable  set  of  skills  and 
a body  of  knowledge  with  related  class  activities  and  course  assignments.  This  content,  using  the  same  textbooks, 
class  lecture  slides,  course  assignments,  and  grading  schema  was  used  in  the  online  course  as  well  as  the  F2F  course. 
Class  activities  were  adapted  for  the  online  environment,  but  used  the  same  content  and  the  same  or  very  similar 
types  of  learner  behaviors.  However,  instructor  presentations  were  replaced  by  website  files  of  course  slides  and 
additional  information  as  required,  and  synchronous  discussions  and  activities  conducted  in  the  classroom  were 
replaced  with  asynchronous  discussions  and  activities  using  the  course  discussion  board  on  the  online  version 
website.  Small  groups  were  formed  as  project  teams  by  the  fourth  week  of  the  sixteen  week  semester  and  were  used 
for  weekly  activities  as  well  as  for  five  major  class  products.  Both  courses  also  had  two  major  assignments  done 
individually  as  well  as  weekly  individual  activities  that  complemented  the  group  work  assignments. 

Learners 

The  subjects  of  this  study  were  36  students  (24  in  the  face-to-face  course  and  12  in  the  online  distance 
learning  course)  enrolled  in  two  different  sections  of  EME  6631,  Management  of  Instructional  Development.  All  of 
the  learners  were  graduate  students,  with  22  of  the  E2E  learners  and  all  of  the  online  learners  majoring  in 
Instructional  Systems.  The  two  non-majors  in  the  E2E  section  were  taking  the  course  as  electives  for  graduate 
programs  in  other  departments  or  colleges.  All  but  one  of  the  online  students  had  taken  at  least  one  online  course 
within  the  past  year.  Three  of  the  online  learners  were  geographically  located  in  the  same  city  as  the  university  while 
the  rest  were  disbursed  throughout  the  southeast,  mid-atlantic,  and  mid-western  states. 

Instructors 

Both  sections  of  the  course  were  taught  by  the  same  instructor  who  had  taught  the  E2E  version  for  the  past 
three  years.  In  the  online  section,  she  was  assisted  by  a teaching  assistant  who  managed  and  monitored  the  course 
website  on  a daily  basis  and  was  the  first  to  receive  questions  on  procedures  or  content.  The  teaching  assistant  had 
taken  the  E2E  version  one  year  before  and  was  completely  familiar  with  the  readings,  in  class  activities,  and 
assignments.  In  addition,  he  had  worked  with  the  instructor  to  develop  the  online  version.  On  occasion  the  teaching 
assistant  referred  questions  of  content  to  the  instructor  who  would  then  post  her  responses  directly  in  the  online 
discussion  board  or  e-mail  the  appropriate  learners.  All  assignments  were  graded  by  the  instructor  for  both  versions. 
The  instructor  led  the  E2E  version  but  the  teaching  assistant  attended  each  class  meeting  to  ensure  his  own 
consistency  in  working  with  the  online  learners. 


Instruments  and  Measurements 

Gathering  and  Coding  of  Interaction  Data 

Two  of  the  F2F  class  meetings  were  observed  and  interactions  coded  at  the  time  and  two  other  class 
meetings  were  videotaped  and  interactions  coded  at  a later  time.  The  online  section  interaction  data  were  obtained 
from  the  archived  data  of  the  small  group  discussion  boards  and  small  group  virtual  chats. 

Learner  communication  patterns  were  coded  into  twelve  categories  based  on  a coding  scheme  developed 
from  Jeong's  model  (2002)  focusing  on  online  debate  and  Poole  and  Fiolmes'  model  (1995)  focusing  on  problem 
solving.  Any  meaningful  statement  between  group  members  is  regarded  as  a communication  fact  and  is  assigned  a 
category  from  the  communication  pattern  model.  The  categorization  is  made  at  the  sentence  level.  An  additional 
category  of  negotiation  was  added  because  negotiation  is  an  important  part  of  the  group  communication  process  in 
the  course,  both  within  groups  and  between  client  and  vendor  groups.  While  Poole  and  Fiolmes  have  a solution 
development  category,  neither  they  nor  Jeong  have  a category  of  interaction  that  includes  sentences  moving  the 
group  toward  consensus  and  involving  statements  of  give  and  take.  The  categories,  therefore,  had  to  be  modified  to 
include  a "negotiation”  communication  fact.  The  twelve  categories,  used  in  the  present  study  are  shown  in  Table  1. 


Table  1: 

Coding  Categories 

Code 

Category 

Explanation 

Example 

0 

State  a proposal 

Initial  statements  proposed  to 
solve  a problem. 

i.e.  “regarding  vendor  suggestion- 1 
think  we  should  mention  we  welcome 
suggestions  to  jump  -start  or  kick  off 
the  project,  as  well  as  suggestions  for 
continued  progress,  motivational 
activities  through  the  winding-down 
period’’ 

1 

Support  a 
proposal 

Support  of  an  initial  proposal 

i.e.  “Audio  make  sense.  If  we’re  aware 
that  people  are  looking  at  a simulate 
screen,  their  hearing  is  free  to  listen  to 
instruction’’ 

2 

Oppose  or 
criticize  a 
proposal 

Oppose,  disagree,  criticize 
other’s  statements 

i.e.  “. . . We  cannot  really  accept  the 
scope  to  be  optimized  because  it  do 
not  fit  for  our  initial  objectives. . .’’ 

3 

Elaboration 

Explanation  of  the  proposal 
without  justifying  or  defending  it 

i.e.  “This  allows  notes  to  be  taken,  etc” 

4 

Simple  agree  or 
disagree 

Statements  of  approval  or 
objection  related  to  other 
statements.  It  usually  only 
involves  very  few  words  and 
does  not  involve  reasoning 
process. 

i.e.  “Good  one,  A. . . should  be  tested 
as  it  will  be  delivered” 

5 

Answer  questions 

Statements  to  answer  the 
questions  for  the  clarification 

i.e.  “User  guide  is  to  assist  learner  in 
self-directed  learning” 

6 

Ask  questions  for 
clarification 

The  statements  to  clarify  the 
content,  the  procedure  of  the 
discussion 

i.e.  “Was  the  CD  to  run  a pilot  prior  to 
load  it  you  intranet?  ( i.e.  “Was  the  CD 
to  run  a pilot  prior  to  load  it  you 
intranet?” 

7 

Negotiation 

Statement  for  consensus  making 
in  group  discussion 

i.e.  “As  your  told  us,  scope  is 
important,  but  what  I am  thinking  is 
that  may  be...” 

8 

Draw  a conclusion 

Statements  attempt  to  make  a 
final  decision  for  a content 
related  topic  for  the  group 
discussion 

i.e.  “O.K.  So  we  can  adjust  our 
estimates  and  staffing  to  reflect  this 
decision.” 

9 

Orientation 

Statements  orient  or  guide 
group’s  process  or  procedure 

i.e.  “Can  we  discuss  the  pilot  now?” 

// 

Non-task 

Any  off-take  statements  not 

i.e.  “Hi,  A,  I haven’t  had  much  luck  in 

related  to  solving  the  problem 
such  as  personal  comments 

getting  us  in -state  tuition.” 

? 

Unknown 

Statements  that  cannot  be  clearly 
transcribed  from  video  tapes  or 
the  statements  two  coders  cannot 
agree  with  each  other  after 
consensus  making. 

NA 

Gathering  and  Coding  of  Student  Perceptions  and  Attitudes 

Two  open-response  anonymous  surveys  were  used  to  measure  student  perceptions  and  attitudes  with  one 
administered  halfway  through  the  semester  (week  8)  and  the  other  during  the  last  week  of  the  course  (week  16).  Five 
categories  of  the  learner  perceptions  toward  the  small  group  team  process  were  measured.  These  questions  asked 
learners  to  1)  describe  their  role  in  the  group  process,  changes  in  roles,  and  their  degree  of  comfort  with  their  roles: 

2)  evaluate  the  process  of  team  discussions  and  methods  for  communication;  3)  evaluate  the  quality  of  the  team 
products;  4)  evaluate  the  feedback  they  received  on  course  activities  and  products;  and  5)  describe  their  preference 
for  course  delivery. 

Learning  Outcomes 

Three  products  developed  by  the  teams  were  used  to  evaluate  learning  outcomes.  These  three  products 
were  the  major  part  of  the  group  work  assignments  for  both  courses  and  all  required  collaborative  group  work. 

These  products  were  the  Estimation  Report,  the  Negotiation  Report,  and  the  Project  Proposal.  The  grading  rubrics 
developed  by  the  instructor  and  used  for  the  previous  three  years  were  used  as  the  evaluation  instrument.  See  Figure 
1 for  an  example. 

FIGURE  1.  Sample  grading  rubric 


Analyses  and  Results 

Interaction  Analysis  and  Results 

Three  activities  of  both  the  online  and  E2F  versions  were  analyzed  by  two  coders.  The  two  coders 
discussed  each  statement  until  they  reached  agreement.  If  an  agreement  could  not  be  reached,  the  statement  was 
categorized  as  unknown  (?).  There  were  altogether  271  F2E  statements  and  848  online  statements. 

Results  for  Question  1 

Are  small  groups  working  in  online  courses  more  task-focused  in  their  communications  than  E2F  course 
small  groups?  Jonassen  and  Kwon’s  (2001)  study  result  on  communication  pattern  is  not  replicated  in  one  (see  Table 
2). Overall,  there  was  no  apparent  difference  in  non-task  statements  between  the  two  different  learning  environments. 
E2F  had  11.8%  non-task  statements  and  online  had  10.0%  non-task  statements,  (see  Table  2 code  //,  Chi-square  = 
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.84  not  significant  at  .05  level)  However,  in  asynchronous  communication  (threaded  discussion  board),  non-task 
statements  make  up  6.1%  (see  Table  3 code  //),  which  is  much  lower  than  those  in  face-to-face  learning  environment 
11.8%.  But,  after  a Chi-square  test,  this  difference  is  not  statistically  significant.  Chi-square  = 5.51  not  significant  at 
.05  level,  but  close  to  the  table  value  5.99) 


Table  2:  Comparison  of  All  Statements  in  Both  Learning  Environments 


Codes 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

// 

7 

All 

Face  to 
face 

statements 

16 

13 

4 

12 

39 

32 

31 

69 

5 

4 

32 

11 

271 

% of 

statements 

4.8 

5.9 

1.5 

4.4 

14.4 

11.8 

11.4 

25.5 

1.8 

1.5 

11.8 

4.1 

100 

On  line 
class 

Combined 

total 

179 

80 

19 

44 

40 

68 

109 

111 

19 

92 

85 

30 

848 

% of 

combined 

total 

21.1 

9.4 

2.2 

5.2 

4.7 

8.0 

12.9 

13.1 

2.2 

10.8 

10.0 

3.5 

100 

Table  3:  Comparison  of  Types  of  Statements  (F2F  and  Online  Threaded  Discussion  and  Virtual  Chat) 


Codes 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

// 

? 

All 

Face  to 
face 

statements 

16 

13 

4 

12 

39 

32 

31 

69 

5 

4 

32 

11 

271 

% of 

statements 

4.8 

5.9 

1.5 

4.4 

14.4 

11.8 

11.4 

25.5 

1.8 

1.5 

11.8 

4.1 

100 

On  line 
class 

discussion 

board 

67 

38 

3 

6 

7 

37 

18 

15 

4 

22 

12 

5 

196 

% of 

statements 

34.2 

19.4 

1.5 

3.1 

3.6 

18.9 

9.2 

7.7 

2.0 

11.2 

6.1 

2.6 

100 

On  line 
class 

discussion 

virtual 

chat 

112 

52 

16 

38 

33 

31 

91 

96 

15 

70 

73 

25 

652 

% of 

statements 

17.2 

8.0 

2.5 

5.8 

5.1 

4.8 

14.0 

14.7 

2.3 

10.7 

11.2 

3.8 

100 

Table  4:  Legend  of  codes  and  categories 


Code 

Category 

Code 

Category 

0 

State  a proposal 

6 

Ask  questions  for  clarification 

1 

Support  a proposal 

7 

Negotiation 

2 

Oppose  or  criticize  a 
proposal 

8 

Draw  a conclusion 

3 

Elaboration 

9 

Orientation 

4 

Simple  agree  or  disagree 

// 

Non -task 

5 

Answer  questions 

7 

Unknown 

11 


Results  for  Question  2 

What  types  of  learner  interactions  are  most  commo  n in  small  groups  in  the  two  learning  environments?  The 
online  environment  had  more  than  four  times  as  many  statements  in  the  category  of  “state  a proposal”  than  did  the 
F2F.  (21.1%  vs.  4.8%,  see  table  2 code  0,  Chi-square  = .32.60,  significant  at  .01  level). There  were  more 
“orientation”  statements  in  the  online  learning  environment  (10.8%  vs.  1.5%,  see  table  3 code  9,  Chi-square=22.56, 
significant  at  .01  level).  The  F2F  environment  had  three  times  as  many  statements  in  the  category  of  “simple  agree 
or  disagree”.  (14.4%  vs.  4.7%,  see  table  2 code  4,  Chi-square=29.81,  significant  at  .01  level)  and  twice  as  many 
statements  in  the  category  of  “negotiation”.  (25.5%  vs.  13.1%,  see  Table  2 code  7,  Chi-square=24.40,  significant  at 
.01  level). 

Analysis  and  Results  of  Learner  Perceptions  and  Attitudes 

Comments  from  the  anonymous  open  response  surveys  were  content-analyzed  by  grouping  common 
statements  into  opinion/feelings  categories  according  to  the  survey  questions  (Patton,  2001).  Then  the  frequencies 
were  computed  for  each  category  and  converted  to  percentages  to  enable  comparisons  between  the  two  groups.  It  is 
important  to  remember  that  there  were  double  the  number  of  learners  in  the  F2F  section,  thus  percentages  in  the  F2F 
section  represent  twice  the  number  of  students  as  do  the  percentages  given  for  the  online  section.  Because  of  the 
small  sample  size,  the  frequencies  were  not  tested  for  significance  but  were  reviewed  for  trends  and  patterns. 

Results  for  Question  3 

Are  online  learners  as  comfortable  in  their  team  roles  as  the  learners  in  F2F  teams?  At  the  midpoint  of  the 
course  the  first  survey  revealed  that  over  half  the  learners  in  both  the  online  and  F2F  courses  indicated  initial 
comfort  with  their  roles  in  their  teams  (online  = 58%  and  F2F  = 68%).  By  the  end  of  the  course  the  results  from  the 
second  survey  showed  that  both  groups  increased  in  comfort  with  their  team  roles  as  the  semester  progressed,  but 
the  online  learners  increased  in  comfort  more  and  became  comparable  to  the  number  of  F2F  learners  feeling 
comfortable  (online  = 75%  and  F2F  = 79%).  Even  though  online  learners  initially  felt  less  comfortable,  they 
finished  the  semester  virtually  as  comfortable  as  the  F2F  class  in  their  group  roles. 

One  interesting  unanticipated  finding  related  to  the  flexibility  of  roles  within  the  group.  While  initially 
about  one  third  of  the  F2F  and  online  learners  indicated  they  were  flexible  in  their  roles  in  group  work,  by  the  end  of 
the  course  67%  of  the  F2F  learners  indicated  that  they  played  both  leader  and  supporter  roles  at  various  times  during 
the  semester,  while  only  33%  of  the  online  learners  said  they  played  various  roles. 

Results  for  Question  4 

Are  online  learners  as  satisfied  with  the  communication  processes  for  group  work  as  the  F2F  learners? 
Through  an  oversight,  feelings  about  the  team  discussion  process  were  gathered  only  at  the  midpoint  of  the  course. 
At  that  time  more  than  2/3  of  the  F2F  learners  were  happy  with  the  team  discussion  process  while  only  a third  of  the 
online  learners  were  satisfied.  See  Table  5 for  the  summary  of  results  on  learner  perceptions  and  attitudes  toward 
their  group  roles,  team  process,  and  group  products. 

Table  5:  Summary  of  Learner  Perceptions  and  Attitudes 


Comfort  with 
Roles 

Elexibility  in 
Roles 

Satisfied  with  Team 
Process 

Satisfied  with  Group 
Products 

F2F 

Midcourse 

68% 

30% 

68% 

74% 

Online 

Midcourse 

58% 

30% 

33% 

67% 

F2F 

End  Course 

79% 

67% 

NA 

74% 

Online 
End  Course 

75% 

33% 

NA 

75% 

Results  for  Question  5 

Comparisons  of  use  of  the  communication  tools  available  to  all  learners  in  both  versions  of  the  course 
showed  that  F2F  learners  actually  made  even  more  use  of  e-mail  than  did  the  online  learners  (F2F  = 94%,  online  = 
83%).  Both  F2F  and  online  learners  made  use  of  the  small  group  discussion  boards,  with  the  online  learners,  not 
surprisingly,  indicated  greater  use  than  the  F2F  learners  (online  = 50%,  F2F  = 39%).  Virtual  chat  was  used  by  only 
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25%  of  the  online  learners  and  none  of  the  F2F  learners.  And  finally,  the  telephone  was  used  by  more  than  two- 
thirds  of  the  online  learners  while  only  a third  of  the  F2F  learners  reported  using  the  telephone.  It  is  important  to 
note  that  all  of  the  online  learners  reporting  use  of  the  telephone  mentioned  scheduling  regular  or  occasional 
conference  calls  with  their  project  teams  (small  groups),  not  individual  telephone  calls  between  two  team  members. 
None  of  the  F2F  learners  mentioned  holding  conference  calls.  The  results  of  learner  use  of  communication  tools  are 
summarized  in  Table  6. 

Table  6:  Summary  of  communication  tool  use 


E-mail 

Small  Group 
Discussion  Bd. 

Virtual  Chat 

Telephone 

F2F 

End  Course 

94% 

39% 

0% 

33% 

Online 
End  Course 

83% 

50% 

25% 

67% 

Results  for  Question  6 

Are  the  learning  outcomes  the  same  for  F2F  and  online  learners?  Evaluation  of  the  three  major  group 
assignments  (estimation  report,  negotiation  reports  and  the  project  proposal)  showed  that  the  products  from  both 
sections  of  the  course  were  equally  satisfactory,  (see  Table  7).  Regarding  learner  perceptions  of  their  group 
products,  at  the  midpoint  of  the  course  the  online  learners  were  less  satisfied  with  the  quality  of  their  group  products 
(67%  of  online  learners  were  very  satisfied  while  74%  of  the  F2F  learners  were  very  satisfied).  By  the  end  of  the 
course,  the  number  of  online  learners  very  satisfied  with  the  group  products  had  increased  and  was  comparable  to 
the  F2F  learners  (online  was  74%,  while  F2F  was  75%).  See  Table  5. 

Table  7:  Learning  outcomes  measured  by  three  assignments 


F2F  Average 

Online  Average 

Estimation  Report  (full  score 
15  points) 

15 

14.33 

Negotiation  Report  (full  score 
5 points) 

5 

5 

Project  Proposal  (full  score  20 
points) 

18.96 

20 

Sum 

38.96 

39.33 

Discussion 

Question  1 

Are  small  groups  working  online  more  task-focused  in  small  group  interactions  than  F2F  small  groups? 

The  present  study  found  no  difference  until  the  threaded  discussion  interactions  were  separated  from  the  Virtual 
Chat  interactions.  Then  the  online  learners  comments  were  more  task  focused  in  the  threaded  discussion  boards.  In 
the  Virtual  Chat,  however,  non-task  comments  were  almost  as  prevalent  in  the  online  environment  as  in  the  F2F 
environment.  Asynchronous  interactions  seems  to  promote  more  time  for  learners  to  reflect  and  develop  indepth 
questions,  comments,  or  responses.  Implications  for  teaching  are  that  F2F  courses  may  benefit  from  some  required 
asynchronous  threaded  discussion  activities  for  small  groups.  This  suggestion  must  be  tempered,  however,  with  the 
need  for  the  instructor  to  pose  questions  or  issues  requiring  reflection. 

Question  2 

What  types  of  interactions  are  most  common  in  small  group  work  in  the  two  different  learning 
environments?  One  interesting  finding  is  that  the  online  environment  has  about  four  times  as  many  statements  in  the 
category  of  “state  a proposal”  than  the  F2F.  (21.1%  vs.  4.8%,  Chi-square=32.60,  significant  at  .01  level).  One 
possible  explanations  is  that  in  the  online  environment,  team  members  have  difficulty  figuring  out  what  has  been 
said,  consequently  they  do  not  respond  to  a proposal  which  causes  the  initiator  to  either  restate  the  proposal  or  to 
make  a new  proposal.  One  solution  may  be  to  provide  online  learners  with  access  to  synchronous  modes  of  audio 
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communication  such  as  Web  4M  or  telephone  conferencing  for  group  organizational  and  planning  meetings  to  help 
learners  be  more  efficient  and  accurate  in  their  group  work. 

The  face  to  face  environment  has  three  times  as  many  statements  in  the  category  of  “simple  agree  or 
disagree’’  (14.4%  vs.  4.7%,  Chi-square=29.81,  significant  at  .01  level).  One  explanation  may  be  that  online 
environment  provides  students  with  more  time  to  reflect  and  thus  more  time  to  elaborate  on  their  opinions.  We  think 
this  finding  has  implication  on  practice  of  course  design.  Course  designer  can  intestinally  put  some  of  F2F  activities 
which  requires  more  time  for  reflection  and  high  order  thinking  on  the  discussion  board  of  the  course  website  to 
promote  learner’s  reflection  process. 

The  face  to  face  environment  has  twice  as  many  statements  in  the  category  of  “negotiation’’  (25.5%  vs. 
13.1% , Chi-square=22.40,  significant  at  .01  level).  It  may  be  that  more  cues  such  as  tones,  gestures,  facial 
expressions  are  needed  to  engage  learners  in  efficient  negotiation.  There  are  more  “orientation’’  statements  in  on-line 
learning  environment.  (10.8%  vs.  1.5%,  Chi-square=22.56,  significant  at  .01  level).  One  explanation  is  that  on-line 
learning  environment  is  less  structured  than  F2F  and  it  requires  more  statements  to  compensate  for  lack  of  social 
cues  and  behavioral  indicators. 

Question  3 

Are  online  learners  as  comfortable  in  their  team  roles  as  learners  in  the  F2F  teams?  In  this  study  only  half 
of  the  online  learners  felt  comfortable  in  their  group  roles  at  the  midpoint  of  the  course  while  over  two  thirds  of  the 
F2F  learners  felt  comfortable  at  that  point.  However,  by  the  end  of  the  course  the  online  learners  were  as 
comfortable  as  the  F2F  learners.  This  finding  may  be  related  to  the  fact  that  the  online  environment  has  fewer  social 
cues  so  it  takes  learners  a longer  period  to  get  to  know  each  other  and  feel  comfortable  working  within  their  teams. 
Also,  online  learners  had  to  experiment  with  the  communication  tools  available  in  the  course,  and  this  required  more 
effort  and  time  for  online  learners  to  determine  what  tools  were  most  effective.  In  contrast,  F2F  learners  did  not 
have  the  same  barriers,  and  began  to  get  to  know  each  other  more  fully  and  easily  during  regularly  scheduled  class 
meetings.  Half  of  the  online  learners  eventually  settled  on  telephone  conferences  to  provide  synchronous  team 
meetings  once  a week.  In  such  communication,  voice  contact  gave  more  social  cues,  as  well  as  help  learners  resolve 
issues  and  make  plans  with  more  ease  and  in  a more  timely  fashion. 

Another  point  to  consider  is  that  online  learners  using  virtual  chat  approximated  F2F  learners  in  the  amount 
of  non-task  statements  they  made,  indicating  that  this  communication  tool  enabled  them  to  exchange  some  “social 
cues’’  information  and  build  feelings  of  intimacy  more  easily.  The  important  element  to  consider  is  that 
synchronous,  as  opposed  to  asynchronous,  communication  seems  to  foster  the  exchange  of  information  that  helps 
learners  begin  to  build  a sense  of  community  and  comfort  within  a small  group. 

Finally,  the  issue  of  role  flexibility  indicated  that  the  F2F  learners  tended  to  play  more  roles  as  the  semester 
progressed,  and  over  three  fourths  expressed  comfort  with  these  roles.  While  three  fourths  of  the  online  learners 
also  expressed  more  comfort  at  the  end  of  the  semester,  they  also  indicated  less  flexibility  in  the  roles  they  played. 
Explanatory  comments  in  the  open  responses  on  the  survey  indicated  that  online  learners  tended  to  feel  that  it  was 
more  expedient  to  settle  into  a role  (whether  leader  or  supporter)  and  continue  to  play  that  role  from  week  to  week. 
This  is  probably  related  to  the  inefficiencies  of  communication  using  the  computer-mediated  tools  and  the  pressure 
to  get  group  work  completed  and  assignments  submitted. 

Questions  4 and  5 

Are  online  learners  as  satisfied  with  the  communication  processes  for  group  work  as  F2F  learners?  Which 
communication  tools  are  most  useful  for  online  learners?  At  the  midpoint  of  the  course  only  one  third  of  the  online 
learners  were  happy  with  the  communication  processes  used  for  small  group  work  while  over  two  thirds  of  the  F2F 
learners  were  satisfied.  Explanatory  comments  indicated  that  crossed  messages,  lack  of  timely  responses, 
conflicting  personal  and  work  schedules  contributed  to  more  online  learners  being  not  completely  satisfied. 

However,  none  of  the  learners  in  either  section  indicated  complete  dissatisfaction;  rather  the  online  learners 
mentioned  problems  in  coordinating  their  efforts  in  a timely  fashion  and  some  occasional  confusion  of  who  would 
do  what. 

In  the  explanatory  comments  related  to  Question  5 on  the  most  useful  communication  tools  for  online 
learners,  two-thirds  of  them  mentioned  how  efficient  and  satisfying  the  use  of  telephone  conferencing  had  been.  The 
audio  portion  enhanced  the  sense  of  team  members  as  individuals  and  how  they  were  responding  to  each  other, 
which  probably  contributed  to  the  sense  of  satisfaction,  too.  While  25%  of  the  online  learners  also  used  the  Virtual 
Chat,  comments  indicated  that  typing  skills  and  problems  in  knowing  who  was  preparing  to  respond  at  any  one  time 
made  the  text -based  Virtual  Chat  less  satisfying  that  the  telephone  conferences.  Here  again,  communicating  in  real 
time  (synchronously)  allowed  learners  to  more  quickly  and  easily  make  plans  and  clear  up  confusions. 
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Dissatisfaction  with  the  team  communication  processes  expressed  in  the  F2F  class  centered  more  on  team  members 
not  showing  up  for  meetings,  getting  work  done  on  time,  or  participating  as  fully  as  expected. 

Of  the  computer- mediated  communication  tools  available  to  both  sections,  the  most  useful  tool  for  both 
delivery  modes  was  definitely  e-mail.  However,  online  learners  mentioned  that  they  did  not  find  it  as  useful  for 
group  discussions  and  team  work  processes  because  of  the  time  it  took  for  everyone  to  respond  and  the  confusion 
resulting  from  crossed  messages.  Instead  over  two -thirds  of  them,  as  mentioned  above,  discussed  the  advantages  of 
telephone  conferencing.  Use  of  the  telephone  was  certainly  not  discouraged  but  neither  was  it  promoted  as  a tool  of 
the  course;  and  certainly  telephone  conferencing  was  beyond  the  course  tools  provided.  However,  two  of  the  online 
teams  had  members  with  access  to  such  conferencing  through  their  work  place.  The  enthusiastic  use  of  telephone 
conferencing  by  the  online  learners  in  this  study  indicates  that  using  such  tools  greatly  enhances  the  communication 
process  for  small  groups.  The  key  seems  to  be  synchronous  exchange  of  information  in  a format  easily  used  by  all 
team  members.  While  not  all  small  group  communications  must  be  synchronous,  comments  indicated  that 
conference  calls  early  in  the  week  helped  the  group  develop  a plan  and  timeline  and  assign  tasks  for  the  next 
product.  Group  members  then  worked  individually,  posting  materials  on  the  group  discussion  board  for  review  or  e- 
mailing  documents  to  specified  team  members.  Text -based  virtual  chat  serve  some  of  the  same  purposes,  but  its 
usefulness  was  diminished  by  the  lack  of  typing  skills  of  the  participants  and  confusions  that  arose  in  waits  for 
response  and  crossed  text  messages. 

Question  6 

Are  learning  outcomes  the  same  for  F2F  and  online  learners?  Evaluation  of  the  work  products  showed  that 
both  online  and  F2F  learners  completed  assignments  on  time  and  with  comparable  high  quality.  While  over  two 
thirds  of  both  online  and  F2F  learners  were  very  satisfied  at  the  course  midpoint  with  the  products  of  their  small 
group  work,  by  the  end  of  the  course  both  groups  had  more  members  now  very  satisfied  with  the  quality  of  their 
products.  The  increase  for  both  types  of  learners  from  mid  to  end  of  course  probably  indicates  the  greater  degree  of 
comfort  they  had  attained  by  the  end  of  the  course  based  on  the  feedback  they  had  gotten  on  earlier  products.  The 
greater  increase  in  the  online  learners  (from  67%  to  75%)  probably  reflects  the  learners’  greater  comfort  in  their 
group  roles,  more  trust  in  their  team  mates,  and  better  facility  in  communicating  online. 

Conclusion 

Small  groups  can  work  effectively  online  and  accomplish  the  same  type  and  quality  of  learning 
achievement  within  one  semester.  However,  online  small  groups  have  to  spend  more  time  orientating  themselves 
and  may  tend  to  stay  in  one  role  throughout  the  semester.  Further,  asynchronous  communication  tools  impede 
learner  and  small  group  efficiency  and  feelings  of  satisfaction.  To  counter  these  tendencies,  instructors  can  try  to 
provide  access  to  synchronous  tools  such  as  Web  4M  or  telephone  conferencing.  However,  if  such  audio  tools  are 
not  available,  encouraging  the  use  of  virtual  chat  will  help  small  groups  get  to  know  each  other  faster  and  develop 
organizational  frameworks  and  make  plans  more  efficiently.  And  F2F  small  group  discussions  may  be  improved  by 
required  online  threaded  discussions.  Such  discussions  can  stimulate  deeper  reflection  and  responses. 
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Abstract 

This  meta-analytic  study  is  unique  and  significant  in  that  all  the  1,341  learners  in  11  studies  completed  the 
Group  Embedded  Figures  Test  (GEFT),  interacted  with  the  same  instructional  module,  and  completed  the  same  five 
criterion  tests  measuring  different  types  of  educational  objectives.  Studies  varied  in  presentation  mode  and  type  of 
independent  variables:  visualization,  rehearsal  and  feedback  strategies.  Within  each  of  the  independent  variables, 
variations  (varied  treatments)  were  examined.  Visualization,  feedback  and  rehearsal  strategies  were  embedded  into 
the  instructional  module  as  a result  of  item  analyses.  Specifically,  the  purpose  of  this  study  was  to  assess  the  effect 
of  the  variations  within  each  independent  variable  in  terms  of  their  effects  in  reducing  the  effect  of  learning  style 
achievement  differences  associated  with  field  independent  (FT)  and  field  dependent  learners  (FD).  One  hundred 
twenty  two  effect  sizes  were  generated.  Results  are  not  only  consistent  with  prior  research  related  to  the 
instructional  effect  of  visualization,  feedback  and  rehearsal  but  provide  the  foundation  for  significant  hypothesis 
generation  related  to  the  instructional  use  of  varied  types  of  visualization,  feedback,  rehearsal  strategies  and 
presentation  mode  in  terms  of  reducing  achievement  differences  on  different  types  of  educational  objectives 
associated  with  individual  learning  styles  (FI  & FD). 

Introduction 

Individual  learning  styles  are  becoming  one  of  the  critical  dimensions  in  the  design  of  instruction  for 
learners  at  all  education  levels.  An  individual’s  learning  style  generally  determines  how  he  or  she  will  proceed  to 
interact  with  the  instructional  content  presented  in  a learning  environment.  In  general,  the  literature  on  learning 
style  research  indicates  that  learners  approach  learning  objects  differently  and  profit  differentially  from  similar  types 
of  instruction  (Jonassen  & Grabowski,  1983).  Implications  from  this  research  indicate  that  differences  in  learning 
style,  field  independence  - field  dependence,  do  affect  achievement  of  different  types  of  learning  objectives  in 
knowledge  acquisition  domains:  facts,  concepts,  rules/principles,  and  procedures  (Alemar,  1992;  Al-Saai,  1993; 
Canelos  & Taylor,  1981;  Couch,  1990;  Dwyer  & Moore,  1991;  Fullerton,  2000;  Joseph,  1987;  Moore  & 
Dwyer,1991,1994,  2000,  2001a,  2001b,  Pollack,  1987). 

Information  Processing 

Upon  perception,  an  event  is  analyzed  from  it  shallow  structural  aspects  to  its  deep  semantic  aspects.  A 
memory  trace  reflects  the  analyses  performed  with  the  deeper  the  analyses,  the  greater  the  degree  of  semantic  or 
cognitive  coding;  hence,  the  more  durable  the  memory  trace  (Moscovitch  & Craik,  1976).  Further  analyses  occur 
through  elaboration  or  enrichment  resulting  in  trace  persistence.  Rehearsal  of  repeated  information  at  the  same  level 
does  not  increase  the  recall  of  information  nor  the  strength  of  the  memory  trace;  deeper  processing  results  in 
increased  levels  of  recall  (Craik  & Lockhart,  1972).  Elaborative  rehearsal,  a function  of  working  memory,  involves 
processes  designed  to  more  deeply  encode  and  store  information  for  later  retrieval  (Craik  & Lockhart,  1972;  Craik 
& Tulving,  1975).  Thus,  elaboration  of  information  facilitates  increased  retention  through  greater  compatibility 
between  the  structure,  rules  and  organization  resulting  in  more  compatibility  with  cueing  for  the  retrieval  of  the 
information. 

Rehearsal 

Consistent  with  the  Information  Processing  Theory,  Dickinson  & O’Connell  (1990)  demonstrated  that 
transformation  of  material  is  essential  for  learning;  reading,  rereading  and  skimming  are  not  considered  to  require  as 
much  transformational  activity  as  organizational  strategies,  the  imposition  of  structure  on  the  material.  Embedded 
questions  within  the  instructional  materials  have  shown  that  rehearsal,  either  prompted  covertly  (thinking  about  the 
answer  to  the  embedded  questions)  or  overtly  (writing/typing  the  answer  to  the  embedded  questions),  does  facilitate 
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increased  achievement  of  the  learning  objectives  (Andre,  1990;  Denner  & Rickards,  1987;  Friedman  & Rickards, 
1981;  Glover,  1989;  King,  1992;  Walker,  1974;  Wang  & Andre,  1991;  Wager  & Mory,  1992;  Watts  & Anderson, 
1971;  Willoughby  & Wood,  1994;  Wilson,  Koran  & Koran,  1974).  Conflicting  research  for  overt  and  covert 
rehearsal  strategies  suggest  that  there  may  be  learning  situations  that  result  in  covert  rehearsal  being  more  effective; 
these  situations  may  be  when  the  instruction  is  perceived  to  be  too  easy  or  the  overt  rehearsal  interferes  with  the 
already  adequate  interaction  with  the  content  presented  (DeKlerk  & DeKlerk,  1978). 

Feedback 

Feedback  within  instruction  has  been  shown  to  be  more  effective  than  no  feedback  (Bangert-Drowns, 

Kulik,  Kulik,  & Morgan,  1991;  Clariana,  1992;  Kulhavy,  1977;  Kulik  & Kulik,  1988).  Information  processing  takes 
place  when  a stimulus  affects  the  learner  and  activates  the  receptors  within  the  brain  to  transmit  information  (Craik 
& Lockhart,  1972).  Feedback  by  itself  does  not  ensure  that  additional  information  processing  is  taking  place  (Wager 
& Mory,  1992).  The  effect  of  feedback  on  achievement  of  field  independent  and  field  dependent  learners  has  not 
been  found  to  create  a significant  difference  within  the  individual  studies  (Al-Saai,  1993;  Canelos  & Taylor,  1981; 
Couch,  1990;  Pollack,  1987).  Comparisons  between  the  studies  is  needed  to  further  investigate  the  effect  of 
different  types  of  feedback. 

Visualization 

Visuals  have  been  shown  to  be  an  important  variable  in  facilitating  achievement  (Baker  & Dwyer,  2001; 
Dwyer,  1978).  While  visuals  have  facilitated  achievement,  visuals  placed  in  positions  identified  through  item 
analysis,  into  locations  within  the  instruction  where  learners  have  difficulty,  have  not  been  enough.  Additional 
instructional  strategies  have  been  placed  in  conjunction  with  the  redundant  visuals  to  assist  learners  in  facilitating 
additional  achievement  (Alemar,  1992;  Al-Saai,  1993;  Canelos  & Taylor,  1981;  Couch,  1990;  Dwyer  & Moore, 

1991;  Fullerton,  2000;  Joseph,  1987;  Moore  & Dwyer,  1991, 1994,  2000,  2001a,  2001b,  Pollack,  1987). 

Field  Independence  - Field  Dependence 

Field  Independence  - field  dependence  is  one  of  the  most  extensively  researched  learning  styles  (Witkin, 
Moore,  Goodenough  & Cox,  1962).  Field  independent  learners  have  been  identified  to  be  superior  to  field 
dependent  learners  when  thinking  operationally  and  recalling  details  in  a prose  passage  (Riding  & Dyer,  1983). 

Field  independent  learners  tend  to  score  higher  on  achievement  tests  due  to  processing  of  text  in  an  analytical 
manner  that  imposes  order  when  necessary  (Hansen,  1983);  field  dependent  learners  tend  to  interact  with  the 
presented  information  using  the  structure  that  is  given.  Implications  from  the  body  of  field  independence  - field 
dependence  research  include  prescriptive  recommendations  based  on  individual  studies.  Examples  of  instructional 
design  prescriptions  to  remediate  or  compensate  for  field  dependent  learners  include:  (1)  provide  abundant  positive 
and  negative  feedback,  (2)  begin  exercises  with  clear  structure,  abundant  cues,  and  consistent  feedback,  and  (3)  have 
learners  paraphrase  the  content  (Jonassen  & Grabowski,  1983).  Structure  created  by  the  field  independent  learner 
will  permit  the  learner  to  process  the  information  presented  to  a level  that  is  recalled.  The  field  dependent  learner 
interacts  with  the  material’s  structure  (Hansen,  1983)  with  the  possibility  that  they  may  not  be  organizing  the 
information  in  a manner  that  promotes  the  deeper  levels  of  processing  necessary  for  later  retrieval.  Additional 
rehearsal  by  the  field  dependent  learner  may  allow  for  deeper  processing  of  the  imposed  structure,  creating  stronger 
memory  traces  and  higher  levels  of  recall. 


Statement  of  the  Problem 

The  purpose  of  this  study  is  to  examine  the  instructional  effects  of  different  types  of  instructional  strategies 
in  terms  of  their  ability  to  facilitate  achievement  of  different  types  of  educational  objectives  in  knowledge 
acquisition  domains:  facts,  concepts,  rules/principles  and  comprehension  for  field  independent  and  field  dependent 
learners  by  summarizing  the  findings  of  a series  of  studies  that  used  the  same  core  instructional  materials  and 
criterion  measures  through  a meta-analytic  analysis.  Specifically,  the  purpose  of  this  meta-analytic  assessment  is  to 
determine  the  effects  of  various  types  of  visuals  (black  & white,  black  & white  shaded,  color  coded,  none),  feedback 
strategies  (independent  of  response,  dependent  upon  response,  no  feedback),  rehearsal  levels  (covert,  prompted 
covert,  overt),  type  of  instructional  presentation  (audio  only,  print  based,  computer  based)  and  type  of  knowledge 
acquisition  domain  (facts,  concepts,  rules/principles,  procedures). 
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Methodology 

Designed  to  accumulate  experimental  results  across  independent  studies  that  address  a related  research 
question,  meta-analysis  is  a set  of  statistical  procedures  that  uses  summary  statistics  from  the  individual  studies  as 
data  points.  Each  data  point  is  assumed  to  represent  an  estimate  of  the  underlying  relationship  within  a population; 
thus,  the  combined  effect  size  for  a meta-analysis  represents  the  population  effect.  Effect  sizes  for  this  study 
represent  the  difference  in  scores  between  field  independent  learners  and  field  dependent  learners  in  an  individual 
study,  experiencing  identical  treatments.  Overall  effect  sizes  for  various  instructional  strategies  or  criterion 
measures  are  the  representative  population  effect  size  for  the  differences  for  field  independent  learners  and  field 
dependent  learners.  Since  all  data  was  reported  as  means  and  standard  deviations  within  the  independent,  individual 
studies.  Hedges  (1982)  and  Hunter  & Schmidt’s  (1990)  technique  for  calculating  the  effect  size  was  employed. 

Since  more  than  five  percent  of  the  individual  studies  failed  to  report  their  unique  reliabilities  for  the  criterion 
measures,  the  corrections  for  variance  due  to  the  artifacts  of  measurement  error  and  restriction  of  range  were  not 
calculated  (Hedges,  1988).  Prior  to  calculation  of  effects  sizes,  the  various  data  points  for  the  means  from  the 
individual  studies  were  examined  for  outliers  using  the  SAMD  technique  developed  by  Huffcutt  & Arthur  (1995). 

No  outliers  were  found  within  the  data  points. 

Eleven  studies  (Appendix  A)  were  identified  as  utilizing  the  Group  Embedded  Eigures  Test  (GEET) 
(Witkin,  Moore,  Goodenough  & Cox,  1962)  for  measuring  field  independence  and  field  dependence  and  the 
instructional  materials  developed  by  Dwyer  (1965).  While  many  studies  employed  the  GEET  for  measuring  field 
independence  and  field  dependence,  only  the  eleven  studies  (n  = 1341)  were  chosen  to  ascertain  more  purely  the 
effect  of  the  various  instructional  strategies  and  techniques  without  the  interference  of  different  criterion  measures; 
one  of  the  leading  criticisms  of  meta-analysis  (Hunter  & Schmidt,  1990;  Rosenthal,  1991).  One  hundred  twenty-two 
effect  sizes  were  calculated  from  the  available  data.  The  means,  standard  deviations  and  sample  sizes  for  the  field 
independent  learners  were  assigned  to  the  experimental  group  and  the  means,  standard  deviations  and  sample  sizes 
for  the  field  independent  learners  were  assigned  to  the  control  group  when  calculating  the  effect  sizes  for  the 
individual  studies’  instructional  strategy  and  knowledge  acquisition  domains.  Analysis  of  variance  was  performed 
to  calculate  the  mean  effect  size  and  to  determine  statistically  significant  differences  of  the  effects  for  each  category: 
visual  characteristics,  feedback  strategy,  rehearsal  level,  presentation  medium,  and  knowledge  acquisition  type. 

Instructional  Unit  and  Criterion  Measures 

The  core  instructional  unit  developed  by  Dwyer  (1965)  contains  approximately  2000  words  about  the 
anatomy  and  functions  of  the  human  heart  with  five  criterion  measures:  comprehension,  drawing,  identification, 
terminology,  and  total.  The  criterion  measures  correspond  to  the  different  knowledge  acquisition  domains:  facts, 
concepts,  rules/principles  and  procedures.  The  instructional  strategies  include  four  levels  of  visuals  (black  & white, 
black  & white  shaded,  color  coded,  none),  three  levels  of  feedback  strategies  (independent  of  response,  dependent 
upon  response,  no  feedback),  three  levels  of  rehearsal  (covert,  prompted  covert,  overt),  and  three  types  of 
instmctional  presentation  (audio  only,  print  based,  computer  based). 

Reliability  information  reports  the  average  Kuder-Richardson  Eormula  20  Reliability  coefficients  from  a 
random  sampling  of  studies  (Dwyer,  1978)  are  0.77  Comprehension  Test,  0.83  Drawing  Test,  0.81  Identification 
Test,  0.83  Terminology  Test,  and  0.92  Total  Test.  The  following  description  of  the  criterion  tests,  adapted  from 
Dwyer  (1978,  pp.  45-47)  illustrates  the  types  of  instructional  objectives  assessed  in  the  studies. 

Drawing  Test 

The  objective  of  the  drawing  test  was  to  evaluate  student  ability  to  construct  and/or  reproduce  items  in  their 
appropriate  context.  The  drawing  test  provided  the  students  with  a numbered  list  of  terms  corresponding  to  the  parts 
of  the  heart  discussed  in  the  instructional  presentation.  The  students  were  required  to  draw  a representative  diagram 
of  the  heart  and  place  the  numbers  of  the  listed  parts  in  their  respective  positions.  Eor  this  test  the  emphasis  was  on 
the  correct  positioning  of  the  verbal  symbols  with  respect  to  one  another  and  in  respect  to  their  concrete  referents. 

Identification  Test 

The  objective  of  the  identification  test  was  to  evaluate  student  ability  to  identify  parts  or  positions  of  an 
object.  This  multiple -choice  test  required  students  to  identify  the  numbered  parts  on  a detailed  drawing  of  a heart. 
Each  part  of  the  heart,  which  had  been  discussed  in  the  presentation,  was  numbered  on  a drawing.  The  objective  of 
this  test  was  to  measure  the  ability  of  the  student  to  use  visual  cues  to  discriminate  one  structure  of  the  heart  from 
another  and  to  associate  specific  parts  of  the  heart  with  their  proper  names. 
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Terminology  Test 

This  test  consisted  of  items  designed  to  measure  knowledge  of  specific  facts,  terms,  and  definitions.  The 
objectives  measured  by  this  type  of  test  are  appropriate  to  all  content  areas  which  have  an  understanding  of  the  basic 
elements  as  a prerequisite  to  the  learning  of  concepts,  rules,  and  principles. 

Comprehension  Test 

Given  the  location  of  certain  parts  of  the  heart  at  a particular  moment  of  its  functioning,  the  student  was 
asked  to  determine  the  position  of  other  specified  parts  or  positions  of  other  specified  parts  of  the  heart  at  the  same 
time.  This  test  required  that  the  students  have  a thorough  understanding  of  the  heart,  its  parts,  its  internal 
functioning,  and  the  simultaneous  processes  occurring  during  the  systolic  and  diastolic  phases.  The  comprehension 
test  was  designed  to  measure  a type  of  understanding  in  which  the  individual  can  use  the  information  being  received 
to  explain  some  other  phenomenon. 

Total  Test  Score 

The  items  contained  in  the  individual  criterion  tests  were  combined  into  a composite  test  score.  The 
purpose  was  to  measure  total  achievement  of  the  objectives  presented  in  the  instructional  unit. 

Results 

By  averaging  the  effect  sizes  and  performing  ANOVA  with  Tukey  follow-up  pairwise  comparisons  to  find 
the  significant  differences,  expected  effect  sizes  to  show  that  the  field  independent  (FI)  learners  perform  higher  than 
the  field  dependent  (FD)  learners.  Each  value  listed  is  the  number  of  standard  deviations  higher  the  field 
independent  learner  would  score  compared  to  the  field  dependent  learner  undergoing  the  same  instructional  strategy 
and/or  knowledge  acquisition  domain. 

Visualization  Results 

Table  1:  Effect  sizes  for  field  independent  participants  compared  to  field  dependent  participants  for  various  visual 
types 


None 

Visual  Type 

T,,  , „ . Black,  White  & 

Black  & White  „ cu  j j 

Gray  Shaded 

Color 

Comprehension 

1.175 

0.701 

0.260 

0.656 

Drawing 

0.551 

0.919 

1.343 

1.239 

Identification 

1.306 

0.793 

1.059 

1.129 

Terminology 

0.989 

0.824 

0.134 

0.876 

Total 

1.236 

0.897 

1.161 

1.380 

Global 

1.051 

0.831 

0.792 

1.072 

In  comparing  the  performance  of  FI  and  FD,  learners  Table  1 illustrated  the  fact  that  field  independent 
learners  are  differentially  affected  when  visualization  is  used  to  complement  instruction  and  differentially  affected 
by  different  types  of  visualization  are  sued  to  facilitate  achievement  of  different  types  of  educational  objectives.  For 
example,  the  effect  size  on  the  comprehension  test  was  1.175  standard  deviations  was  realized  in  favor  of  the  FI  over 
FD  learners  when  both  groups  received  verbal  instruction  without  visuals.  When  varied  types  of  visualization  were 
used  to  complement  the  verbal  instruction,  effect  sizes  on  all  criterion  measures  decrease  indicating  that  the  use  of 
visualization  may  be  considered  an  important  instructional  variable  in  reducing  the  effect  of  learning  style 
differences  between  FI  and  FD  types  learners.  It  is  also  interesting  to  note  the  differences  between  the  color-coded 
and  the  black  and  white  treatments  on  the  various  criterion  measures.  Also  worthy  of  note  is  the  that  the  black, 
white  and  gray  shaded  treatment  was  the  most  effective  in  reducing  achievement  differences  between  FI  and  FD 
learners  on  both  the  terminology  and  comprehension  criterion  measures. 

Feedback  Results 

Feedback  strategies  were  coded  as  no  feedback  conditions  within  the  instruction,  independent  feedback 
conditions  (feedback  that  occurs  regardless  how  the  learner  responds  to  questions  or  other  prompts  for  a response; 
e.g.,  the  answer  that  appears  on  the  next  page),  and  dependent  feedback  conditions  (feedback  that  is  developed 
dependent  upon  the  response;  e.g.,  response  to  learner  telling  them  information  related  to  their  answer). 
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Table  2 : Effect  sizes  for  field  independent  participants  compared  to  field  dependent  participants  for  various 
feedback  types 


None 

Feedback  Type 
Independent 

Dependent 

Comprehension 

0.650 

1.351 

0.376 

Drawing 

1.106 

0.876 

0.212 

Identification 

1.040 

0.692 

0.221 

Terminology 

0.783 

1.001 

0.182 

Total 

1.155 

1.116 

0.283 

Global 

0.958 

1.007 

0.255 

Table  2 illustrates  the  achievement  differences  between  FI  and  FD  learners  when  they  receive  no  feedback 
while  progressing  through  their  instruction.  It  also  illustrates  that  the  different  feedback  types  differentially  effect 
achievement  on  the  different  criterion  measures  with  the  dependent  feedback  type  being  most  instrumental  in 
reducing  achievement  differences  between  FI  and  FD  learners.  It  also  dramatically  presents  the  effect  size 
differences  realized  between  the  independent  and  dependent  feedback  strategies  and  the  potential  significance  that 
may  be  realized  in  reducing  learning  differences  when  these  two  feedback  strategies  are  employed  and  verifies  the 
fact  that  different  feedback  strategies  are  not  equally  effective  in  reducing  achievement  differences  between  FI  and 
FD  learners.  It  is  also  important  to  notice  that  the  interactivity  instigated  between  the  field  dependent  learners 
receiving  the  dependent  feedback  and  the  content  was  effective  across  all  criterion  measures  requiring  different 
levels  of  information  processing.  Apparently  dependent  type  feedback  activities  alerting  the  FD  learners  to  the 
critical  content  to  be  learned  enables  them  to  interact  and  achieve  at  levels  similar  to  those  achieved  by  the  FI 
learners. 

Rehearsal  Results 

The  type  of  rehearsal  has  been  shown  in  other  studies  to  facilitate  recall  of  information  by  deepening  the 
level  of  encoding  through  information  processing  (Craik  & Lockhart,  1972,  Craik  and  Tulving,  1975). 


Table  3 : Effect  sizes  for  field  independent  participants  compared  to  field  dependent  participants  for  various 
rehearsal  types 


Covert 

Rehearsal  Type 
Prompted  Covert 

Overt 

Comprehension 

0.675 

0.325 

0.788 

Drawing 

1.251 

0.403 

0.611 

Identification** 

1.258 

0.408 

0.359 

Terminology 

0.871 

0.373 

0.588 

Total 

1.316 

0.773 

0.697 

Global* 

1.086 

0.509 

0.609 

* p < 0.05  Statistically  significant  difference  for  covert  rehearsal  compared  to  both  prompted  covert  and  overt 
rehearsal  strategies. 

**  p < 0.05  Statistically  significant  difference  for  covert  rehearsal  compared  to  overt  rehearsal  for  the  identification 
test. 

In  comparing  the  effect  sizes  related  to  rehearsal  type  (covert,  prompted  covert,  overt),  it  is  apparent  that 
the  prompted  covert  rehearsal  strategy  offers  the  most  potential  in  reducing  learning  differences  between  FI  and  FD 
learners  among  the  three  rehearsal  types.  Results  indicate  that  the  information  processing  instigated  by  the 
prompted  covert  rehearsal  strategies  were  instrumental  across  all  criterion  measures  in  reducing  performance 
differences  between  the  FI  and  FD  learners.  Covert  rehearsal  strategies,  in  general,  were  the  least  effective  type  of 
rehearsal  in  reducing  achievement  differences  amo  ng  the  different  criterion  measures.  It  is  also  apparent  that 
different  types  of  rehearsal  strategies  are  differentially  effective  in  reducing  achievement  differences  between  FI  and 
FD  learners  and  are  differentially  effective  in  reducing  achievement  differences  on  the  different  criterion  measures. 

ANOVA  results  also  indicate  that  across  all  domains  of  knowledge  acquisition,  that  any  type  of  prompting 
for  rehearsal  facilitates  increased  processing  and  increased  recall  on  the  criterion  measures;  both  prompted  covert 
rehearsal  and  overt  rehearsal  strategies  decreased  the  difference  in  achievement  by  field  independent  learners 
compared  to  field  dependent  learners  as  compared  to  the  learners  rehearsing  covertly  with  no  prompt  to  rehearse  the 
information  at  a deeper  level. 
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Presentation  Mode 

Computer  based  instruction  assumes  to  be  more  effective  due  to  increased  ability  to  program  dependent 
feedback  conditions  or  additional  rehearsal  strategies.  Table  4 presents  the  results  for  the  various  presentation 
formats  for  the  instructional  unit.  Audio  only  presented  slide  images  on  a screen  while  an  audiotape  played  the 
narration  of  the  instructional  script.  Computer  based  instruction  presented  screen  with  the  same  images  and 
materials  as  presented  in  a print  base/paper  based  instructional  booklet.  Within  the  computer  based  instructional 
units,  some  treatments  used  the  same  independent  of  response  feedback  strategy  as  the  paper  based  version;  placing 
the  correct  answer  on  the  top  of  the  next  slide/page.  A common  comment  within  the  computer  based  research 
situations  is  the  statement  that  the  computer  base  may  be  more  motivational  and  lead  to  higher  scores  (Fullerton, 
2000;  Joseph,  1987). 


Table  4:  Ejfect  sizes  for  field  independent  participants  compared  to  field  dependent  participants  for  various 
presentation  types 


Audio 

Presentation  Type 
Computer  Base 

Paper  Base 

Comprehension 

0.652 

0.712 

0.667 

Drawing 

0.818 

1.015 

1.230 

Identification 

0.531 

1.509 

1.074 

Terminology 

0.572 

1.240 

0.748 

Total 

0.685 

1.270 

1.240 

Global* 

0.667 

1.157 

1.002 

* p < 0.05  Statistically  significant  difference  for  type  of  presentation.  Follow-up  Tukey  pairwise  comparisons 
failed  to  find  the  difference. 

Table  4 examines  the  effect  sizes  resulting  from  methods  of  presentation  type  (audio,  computer,  and  paper). 
Results  indicate  that  the  audio  presentation  format  possesses  the  greatest  potential  for  reducing  the  achievement 
effect  of  learning  styles  between  FI  and  FD  learners.  This  result  may  have  its  foundation  in  the  fact  that  the  audio 
presentation  enabled  the  FD  learners  to  hear  the  proper  pronunciation  of  the  information  being  presented  and  were 
better  able  to  make  the  proper  associations  necessary  to  assimilate  the  information  and  move  it  from  short  to  long 
term  memory.  These  results  are  interesting  in  that  the  audio  presentations  were  externally  paced  and  should  not 
have  theoretically  been  competitive  with  the  self-paced  computer  and  paper  oriented  presentations.  Also  worth 
noting  is  the  lack  of  variability  within  presentation  types  in  terms  of  reducing  differences  between  learners  identified 
as  FI  and  FD  on  the  varied  criterion  measures.  These  results  offer  implications  for  justifying  the  use  of  streaming 
audio  in  the  new  electronic  delivery  systems. 


Discussion 

Analysis  of  the  results  within  the  different  studies  were  consistent  with  current  research  indicating  that 
carefully  designed  and  positioned  visualization,  rehearsal  and  feedback  strategies  can  significantly  improve  learner 
achievement.  The  purpose  of  this  study  was  to  systematically  examine  the  effects  of  different  types  of  visualization, 
rehearsal  and  feedback  strategies  and  their  impact  upon  FI  and  FD  learners  as  they  interact  with  instructional  content 
designed  to  facilitate  achievement  on  five  different  types  of  knowledge  acquisition  domains.  In  analyzing  the  effect 
size  related  to  type  of  visualization  the  results  indicate  that  in  general  visualization  can  reduce  achievement 
differences  attributed  to  learning  style  of  FI  and  FD  learners.  Results  also  indicate  that  within  the  domain  of 
visualization,  different  types  of  visualization  function  differentially  in  reducing  achievement  differences  and  also 
function  differently  in  reducing  differences  on  the  different  types  of  criterion  measures.  It  was  also  found  that 
different  feedback  and  rehearsal  strategies  also  functioned  differently  in  reducing  achievement  differences 
associated  with  the  different  learning  styles  of  FI  and  FD  learners.  Differences  in  the  variation  within  treatments 
were  also  realized  in  terms  of  their  ability  to  reduce  achievement  differences  of  the  different  criterion  measures. 
Method  of  presentation  was  also  found  to  be  an  important  variable  in  reducing  achievement  differences  associated 
with  FI  and  FD  learners  on  the  different  criterion  measures. 

These  observations  substantiate  the  fact  that  in  attempting  to  maximize  learning  one  cannot  simply  talk  in 
terms  of  adding  visualization,  feedback,  or  rehearsal  strategies  to  an  instructional  unit  in  an  attempt  to  facilitate 
achievement.  Variations  within  the  different  types  of  visualization,  rehearsal  and  feedback  strategies  function 
differently  - some  functioning  to  reduce  achievement  differences  between  FI  and  FD  learners  and  others  having  very 
little  influence.  It  is  also  important  to  realize  that  since  variations  exist  within  the  independent  variable  types,  it  is 
apparent  that  one  cannot  simply  talk  about  reducing  achievement  differences  between  FI  and  FD  learners  in  the 
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abstract.  Achievement  variables  (types  of  educational  objectives  to  be  facilitated)  need  to  be  identified  precisely  so 
that  prescriptiveness  can  begin  to  evolve  as  a strategy  for  designing  varied  types  of  learning  environments,  which 
might  eventually  lead  to  the  reduction  of  achievement  differences  associated  with  differences  in  learning  style. 

By  averaging  the  effect  sizes  across  all  instructional  strategies  and  knowledge  acquisition  domains,  effect 
sizes  indicate  that  field  independent  learners  do  achieve  higher  scores  on  achievement  tests.  These  gaps  in 
achievement  may  be  closed  by  prompting  the  field  dependent  for  deeper  processing  of  the  information  through 
rehearsal  as  evidenced  by  statistically  significant  differences  for  effects  with  rehearsal  levels  on  the  identification 
test  and  across  all  knowledge  acquisition  domains.  Osman  & Hannafin  (1994)  found  that  orienting  questions  of  a 
conceptual  nature  improved  both  factual  learning  and  problem  solving  and  activated  prior  knowledge  to  promote 
deeper  processing,  which,  in  turn,  promoted  more  meaningful  learning.  This  study  found  similar  findings  for 
prompting  rehearsal  to  facilitate  a decrease  in  the  difference  between  achievement  of  field  independent  and  field 
dependent  learners.  This  prompting  may  be  questions  as  used  by  Reynolds,  Standiford  & Anderson  (1979)  when 
testing  at  the  factual  level:  questions  used  within  the  instruction  resulted  in  higher  achievement  since  they  caused 
learners  to  interact  with  the  material. 

The  implication  for  instructional  designers  is  that  visuals  (Dwyer,  1978),  feedback  (Clariana,  1992),  and 
medium  of  presentation  are  not  individually  effective  enough  to  facilitate  closing  the  gap  between  achievement  of 
field  independent  learners  and  field  dependent  learners.  Prompting  the  learners  for  additional  rehearsal  to  deepen 
the  level  of  processing  to  facilitate  higher  levels  of  retrieval  and  recall  is  necessary. 

Future  research  implications  for  research  include  a need  for  additional  study  of  dependent  feedback 
conditions  as  compared  to  no  feedback  strategy  or  independent  feedback  strategy  since  results  indicate  that  the 
dependent  feedback  condition  is  a low  as  one-tenth  of  the  difference  found  for  the  other  strategies.  A meta-analysis 
of  field  independent  - field  dependent  studies  using  a variety  of  criterion  measures  that  examine  the  use  of  feedback 
strategies  is  recommended.  The  significant  difference  indicated  by  ANOVA  for  the  presentation  type  indicates  that 
the  role  of  the  presentation  medium  may  also  be  a factor  in  subsequent  performance.  Additional  study  of  method  of 
presentation  is  also  recommended. 

The  purpose  of  this  study  was  to  support  and  determine  recommendations  for  instructional  designers  to 
facilitate  a more  equal  performance  by  learners  that  are  identified  as  field  independent  and  field  dependent.  The 
results  do  provide  support  for  the  need  to  include  prompting  techniques  for  rehearsal  within  the  instructional 
materials.  The  findings  of  this  study  also  provide  the  foundation  for  systematically  generating  a multitude  of 
research  hypotheses  dealing  with  the  systematic  examination  of  the  instructional  effectiveness  of  variations  within 
different  variable  domains  as  they  relate  to  different  types  of  educational  objectives. 
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The  effect  of  different  levels 

1992 

244 

Drawing, 

bw,  bws,  cc,  fi. 

Moderate  color  treatment 

of  coding  (color  vs 
black/gray/white)  in 
facilitating  achievement  in 

Identification, 

Terminology, 

Comprehension, 

fd,  fn,  covert, 
paper 

participants  have  higher 
achievement  than 
moderate  black,  gray  and 

learning  facts,  concepts,  and 
generalizations  with  students 
of  different  learning  styles 

Total 

white  treatment. 

Al-Saai 

The  effect  of  different 

1993 
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Eield  independent 

visualized  treatments  on  field 

Identification, 
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independent  and  field 

Terminology, 

pcovert,overt 

achievement  than  field 

dependent  students  at  the 
University  of  Qatar 

Comprehension,  paper 
Total 

dependent  participants  in 
treatment  with  visual  and 

text  and  treatment  with 
overt  rehearsal. 

Canelos,Tayl  A networking  information 
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bw,  c,  fi,  fd. 

Eield  independent 

or 

processing  strategy  and  the 
learning  of  field  dependents 
receiving  visual  instructional 
information 

covert,  pcovert,  participants  have  higher 
audio  achievement  than  field 
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terminology  tests. 

Couch 

The  effects  of  imagery 
rehearsal  strategy  and 
cognitive  style  on  learning  of 
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Eield  independent 
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tests  with  students  of  different  Comprehension  paper  achievement  than  field 

field  dependence  levels  dependent  participants  for 
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color  coded  treatments 
have  higher  achievement 
than  those  exposed  to 
black  and  white 

treatments. 

bw,  fi,  fn,  fd.  Field  independent  and 
covert,  cmpt  field  neutral  participants 
have  higher  achievement 
than  field  dependent 
participants. 


Fullerton  The  interactive  effects  of  field  2000  98  Total 
dependence-independence  and 
internet  document 
manipulation  style  on  student 
achievement  from  computer- 
based  instruction 


Dwyer, 

Moore 


Joseph 


Moore, 

Dwyer 


Moore, 

Dwyer 


Field  dependence/  1987 

independence  and  the 
mathemagenic  effects  of 
observable  visual  imagery 
rehearsal  for  learning  from 
computer-presented  text 


The  effect  of  cognitive  style  1994 
on  test  type  (visual  or  verbal) 
and  color  coding 


54  Drawing,  bw,  fi,  fd. 
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Terminology,  cmpt 
Comprehension, 

Total 


183  Drawing,  bw,  cc,  fi,  fn. 

Identification,  fd,  covert. 
Terminology,  paper 
Comprehension, 

Total 


Field  independent 
participants  have  higher 
achievement  on  the  total 
test  with  visual  images 
that  those  that  had  a 
visual  strategy  present. 
Field  dependent 
participants  have  higher 
achievement  than  those 
that  received  no  strategy. 
Field  independent 
participants  have  higher 
achievement  than  other 
participants  on  color- 
coded  treatment  for  the 
drawing  test,  and  all 
treatments  for  the 
remaining  criterion 
measures . 


The  effect  of  color  coded  1991  119  Identification,  bw,  cc,  fi,  fn, 
information  on  students'  level  Terminology,  fd,  covert, 

of  field  dependence  Comprehension  paper 


Field  independent 
participants  have  higher 
achievement  than  field 
dependent  participants  for 
black  and  white  and  color- 
coded  treatments. 
Participants  exposed  to 
color  coded  treatments 
have  higher  achievement 
than  those  exposed  to 
black  and  white 
treatments. 


26 


Author 


Moore, 

Dwyer 


Moore, 

Dwyer 


Pollack 
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Year  N Criterion  Independent  Outcome 

measures  variables 


bw,  cc,  cfid.  Field  independent 

covert,  paper  participants  have  higher 

achievement  than  field 
dependent  participants. 
Participants  exposed  to 
color  coded  visuals  have 
higher  achievement  than 
those  exposed  to  black 

and  white. 

The  relationship  of  field  2001  55  Drawing,  bw,  c,  ft,  fn,  fd.  Field  independent 

dependence  and  color  coding  Identification,  covert,  paper  participants  have  higher 

to  male  student  achievement  Terminology,  achievement  than  field 

Comprehension,  dependent  participants. 

Total 


The  effect  of  field  dependence  2000  126  Total 

and  color  coding  on  female 

student  achievement  of 

different  educational 

objectives 


The  use  of  interactive  video  to  1987  118  Drawing,  c,  ft,  fn,  fd.  Field  independent 

examine  the  interactive  effect  Identification,  covert,  pcovert,  participants  have  higher 

of  cognitive  style  and  Terminology,  overt,  audio  achievement  that  field 

structural  organizers  on  Comprehension,  dependent  participants  for 

learning  Total  the  total,  drawing, 

terminology,  and 
comprehension  tests. 
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Abstract 

The  categorization  of  types  of  discussion  is  used  to  examine  patterns  of  online  interactions  in  asynchronous 
communications  in  graduate  courses  delivered  at  two  public  universities.  Online  interaction  patterns  are  reviewed 
to  look  at  levels  of  initiation  and  response.  Online  communications  patterns  are  identified  based  on  roles  exhibited 
by  online  participants.  The  structure  of  the  communication  spaces  and  implications  for  online  discussions  will  be 
addressed. 


Introduction 

The  availability  of  new  technologies  such  as  email,  listservs,  and  computer  conferencing  has  begun 
facilitating  new  ways  of  communicating  in  instructional  settings.  Several  communication  tools  now  exist  to  support 
asynchronous  discussions  that  can  be  incorporated  into  existing  classes.  These  tools  support  messaging  between 
individuals  and  facilitate  the  ability  of  participants  to  read  and  respond  to  messages  or  to  add  their  own  new 
messages  to  which  others  can  respond.  Discussions  can  take  place  among  individuals  in  widely  dispersed 
geographic  locations  or  among  persons  unable  to  participate  in  a discussion  at  a specific  time. 

In  these  new  instructional  environments,  technology  can  be  used  for  content  transmission  or  as  a 
communication  support  tool,  or  these  two  roles  can  be  combined,  as  they  often  are  in  online  learning,  to  support 
educational  activities  (Benbunan-Fich  & Hiltz,  1999).  E-learning  tools  can  create  systems  that  allow  students  to 
exchange  messages  and  participate  in  discussions  in  an  organized  way  (Hiltz  & Wellman,  1997).  The  challenge  is  to 
design  pedagogically  effective  learning  environments  in  the  online  world  in  order  to  enhance  the  quality  of 
education  (Althaus,  1997). 

Some  researchers  have  identified  advantages  and  disadvantages  in  implementing  computer- mediated 
discussions  as  an  instructional  tool  (Hilt,  Johnson,  & Turoff,  1986;  Straus  & McGrath,  1994;  Walther,  1996).  While 
computer-mediated  group  interactions  may  be  more  focused  on  tasks  and  less  on  personal  interactions,  they  also 
may  result  in  greater  processing  time  and  create  difficulty  in  consensus  building.  Studies  have  found  more  equal 
participation  and  more  idea  generation  in  computer-mediated  environments  as  participants  have  more  equal 
“speaking”  time.  Lack  of  structure  in  the  online  context  and  a reduced  likelihood  of  leadership  emerging  are  other 
disadvantages  pointed  to  in  the  literature.  Hiltz  et.  al.  (1986)  indicate  that  a leadership  void  may  inhibit  consensus 
building  and  organization  of  a group  in  approaching  a problem.  Farnham,  Chesley,  McGhee,  Kawal,  & Landau 
(2000)  found  that  enhancing  structure  in  an  online  environment  contributed  to  higher  levels  of  consensus  and  better 
decision-making  in  group  activities.  Some  contend  that  asynchronous  interaction  improves  in-depth  reflection  and 
topic  development  (Harasim,  1990). 

Hammond  (1997)  looked  at  the  use  of  online  learning  for  professional  development,  focusing  particularly 
on  the  usefulness  of  the  medium  for  discussion-oriented  activities.  While  concluding  that  the  medium  can  be  used 
effectively,  Hammond  also  pointed  to  a number  of  issues  to  consider  in  developing  online  discussions.  Issues  such 
as  acquiring  sufficient  technical  skills,  constraints  on  writing  skills,  reticence,  and  access  to  technology  were  noted, 
however,  the  author  discussed  in  more  detail  the  difficulty  of  maintaining  the  debate  and  structuring  the  discussion 
so  as  to  provide  openness  and  at  the  same  time  control  over  learning.  While  the  instructor  can  initiate  a discussion, 
there  may  be  little  control  over  who  responds  and  there  may  be  a sense  of  being  removed  from  the  interactions.  The 
online  tools  provide  convenience  for  participation  when  and  where  the  individual  likes  and  thus  increases 
opportunities  for  contributions.  At  the  same  time,  adding  structure  may  reduce  flexibility  and  the  sense  of  being 
“distant”  may  contribute  to  delays  in  participation. 

Muscella  and  DiMauro  (1995)  discuss  maintenance  of  online  debates,  suggesting  that  an  assertion  followed 
by  personal  experience  and  a statement  of  belief  will  trigger  a response  to  a message.  They  also  suggest  starting  with 
non-controversial  topics  as  participants  learn  together  in  order  to  reduce  contentiousness  and  build  a comfortable 
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learning  environment.  Use  of  the  online  medium  for  interactive  discussions  and  collaborative  learning  has  been 
recommended.  Such  strategies  requires  student  to  take  a more  active  role  in  the  learning  process  (Lindeman,  Kent, 
Kinzie,  Larsen,  Ashmore,  & Becker,  1995). 

Some  researchers  have  analyzed  online  discourse  from  listservs  and  web-based  discussion  groups  and 
looked  at  interaction  patterns  (Chase,  Macfadyen,  Reeder,  & Roche,  2002).  Chase  et.  al.  identified  nine  emergent 
themes  from  their  sample  of  text  discourse  from  an  online  class.  First,  an  online  culture  developed  reflecting  the 
values  of  the  developer  of  the  web  environment.  That  culture  was  maintained  by  the  guidelines  created,  and  by  the 
facilitators  and  participants.  Second,  formal  and  informal  participation  was  affected  in  the  online  environment  and 
distinct  communication  pattern  differences  were  apparent  between  the  two.  Third,  individuals  varied  with  their  level 
of  comfort  in  online  discourse.  Fourth,  individuals  created  their  own  online  identity.  Fifth,  technical  issues  and 
formatting  influenced  communication.  Sixth,  participant  expectations  of  the  course,  the  instructor,  and  the  medium 
influenced  the  environment.  Seventh,  facilitator  expectations  also  affected  the  learning  environment.  Eighth, 
differences  in  communication  related  to  the  use  of  academic  discourse  versus  the  telling  of  stories  or  narratives  were 
observed  and  created  variation  in  participation  in  online  debate.  Ninth,  explicit  and  implicit  assumptions  about  time 
were  evident. 

Another  way  to  look  at  online  discourse  is  to  look  at  the  types  of  communication  or  levels  of 
communication  that  occur.  Research  on  questioning  and  discussion  (Dillon,  1990,  1994;  Roby,  1988)  has  some 
interesting  implications  for  examining  online  communication. 

Categorization  of  Discussion  Behavior 

For  instructional  purposes,  it  is  important  not  only  that  students  communicate  in  the  online  environment, 
but  that  we  examine  their  patterns  of  that  communication.  Research  on  questioning  and  discussion  (Roby,  1988; 
Dillon,  1990,  1994)  has  provided  a system  for  categorizing  types  of  communication  based  on  five  patterns. 

Initiating  patterns  of  communication  include  such  things  as  stating  an  opinion  or  insight  to  get  the  conversation 
started,  formulating  a question  to  open  a discussion  topic,  injecting  a new  insight  or  new  information  into  an 
ongoing  discussion,  restarting  a discussion  by  suggesting  a new  approach  or  idea  not  previously  discussed,  or  asking 
for  an  opinion  from  someone  who  is  not  actively  participating  in  the  discussion.  In  a supporting  communication 
pattern,  the  student  may  share  evidence  to  support  a position,  provide  an  example  of  a concept  being  discussed,  ask 
for  clarification,  restate  a position  in  different  words,  or  introduce  a nuance  that  enriches  the  original  information. 
Examples  of  challenging  communication  patterns  include  simple  statements  of  disagreement,  offering  different 
opinions,  or  correcting  facts.  Summarizing  patterns  occur  when  a participant  states  in  a concise  way  the  essence  of 
someone  else’s  remarks  or  condenses  a whole  series  of  remarks  from  different  participants  into  a concise  statement. 
Monitoring  is  defined  as  statements  that  keep  the  group  on  task  and  focus  the  discussion  on  the  topic. 

Context 

This  study  investigated  students’  use  of  one  asynchronous  communication  tool  used  as  part  of  a “hybrid” 
class  that  included  face-to-face  meetings  as  well  as  web-enhanced  instructional  activities.  The  researchers  analyzed 
communication  patterns  used  during  asynchronous  discussions  about  case  studies  and  also  examined  the  type  of 
interactions  that  occurred  among  the  students. 

Two  graduate  courses  at  two  public  universities  (Midwest  and  Southeast)  were  the  focus  of  a study  to 
examine  communication  patterns  in  online  asynchronous  discussions.  One  course  was  in  curriculum  and  instruction 
(Case  1)  and  the  other  in  counseling  (Case  2).  Students  in  the  curriculum  and  instruction  course  studied  concepts  in 
program  evaluation  and  designed  evaluation  plans  for  educational  programs.  Students  in  the  counseling  course 
studied  ethical  and  legal  aspects  of  counseling.  Students  in  both  courses  worked  in  teams,  used  case  studies,  and  had 
projects  and  assigned  readings.  The  two  courses  were  traditional  courses  with  web -enhancements,  including  web 
pages,  chat  rooms,  and  the  use  of  asynchronous  communication  tools.  An  electronic  bulletin  board  (e-board)  was 
used  to  facilitate  group  discussions.  Through  review  of  transcripts  from  the  e-board,  the  researchers  looked  at  how 
students  used  the  e-board  to  support  online  discussions,  what  patterns  of  communication  emerged  and  what  types  of 
interactions  occurred. 

Methods 

The  researchers  used  a qualitative  content  analysis  approach.  Data  were  collected  from  transcripts 
generated  on  the  e-board  that  students  used  during  the  online  discussion  component  of  the  courses.  The  identified 
categories  of  communication  were  used  to  code  the  transcripts  from  four  case  study  discussions.  Initiating  (I), 
supporting  (SP),  challenging  (C),  summarizing  (SM),  and  monitoring  (M)  communications  were  coded  using  the 
identified  definitions  as  a way  to  look  at  patterns  in  the  discussion.  Reponses  were  also  coded  as  an  initial  posting 
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(IP),  response  to  a post  (RP),  reply  to  a response  (RR),  or  reply  to  a reply  (RR#)  to  look  at  interaction  patterns 
among  the  students.  In  an  initial  posting,  a student  started  a thread  in  the  online  discussion  by  posting  a message.  In 
a response  to  a post,  the  student  responded  to  the  initial  message  posted  in  a given  thread.  A reply  to  a response 
occurred  when  a student  replied  to  the  responses  on  an  initial  posting.  A reply  to  a reply  occurred  when  the  student 
replied  to  one  of  the  replies  made  by  another  student.  In  the  discussion  thread,  the  patterns  might  appear  as  below  in 
Figure  1.  The  researchers  referred  to  this  as  the  Initiate-Response-Reply  Framework  (IRR). 

Figure  1.  Response  patterns  in  a discussion  thread:  The  IRR  Framework 

Level  1:  Initial  Posting  (IP) 

Level  2:  Response  to  the  Post  (RP) 

Response  to  the  Post  (RP) 

Level  3:  Reply  to  the  Response  (RR) 

Response  to  the  Post  (RP) 

Reply  to  the  Response  (RR) 

Level  4:  Reply  to  the  Reply  (RRl) 

Reply  to  the  Reply  (RRl) 

Reply  to  the  Reply  (RR2) 

Reply  to  the  Reply  (RR3) 

Higher  levels  of  responses  and  replies  (level  4 is  high  and  level  1 is  low)  mean  more  interactions  among  the 
participants  in  the  online  discussion.  To  summarize,  the  transcript  data  from  the  e-board  discussions  that  occurred 
during  four  case-study  assignments  was  categorized  in  two  ways:  (1)  type  of  communication  (based  on  identified 
definitions),  and  (2)  interaction  level  observed  (the  IRR  framework) 

Findings 

Three  sets  of  findings  are  reported  below.  First  the  results  of  examining  the  comments  on  the  e-board  based 
on  the  communication  patterns  as  defined  are  reviewed.  Second,  a summary  of  the  interaction  pattern  evident  in  the 
postings  and  responses  is  presented.  Third,  additional  patterns  in  the  content  of  the  discussions  and  the  use  of  the  e- 
board  space  that  emerged  during  the  analysis  will  be  reported. 

Patterns  of  Communication 

One  primary  pattern  in  both  cases  was  that  of  initiating  responses.  Case  1 had  a high  frequency  of  initiating 
communication,  while  initiating  behavior  ranked  third  in  Case  2.  Examples  of  initiating  behavior  are  noted  below. 

Initiating  Example  1: 

What  seems  as  such  a distinct  difference,  formative  as  process  and  summative  as  final;  I find  (as  I usually 
do)  gray  areas.  Principal  evaluations  of  teachers.  Eormative,  right?  They  go  on  year  after  year  in  attempt  to 
help  the  teacher  become  better  and  better.  But,  usually  the  evaluation  is  done  for  a specific  lesson,  which 
the  principal  looks  at  in  detail  and  gives  a summative  report. 

Initiating  Example  2: 

Does  a key  audience  have  to  be  one  individual  group  or  could  it  be  multiple  groups? 

Both  cases  exhibited  high  supporting  patterns  of  communication.  Supporting  responses  were  the  most 
prevalent  type  of  communication  overall  and  in  each  of  the  cases.  Examples  of  comments  coded  as  supporting  are 
noted  below. 

Supporting  Example  1: 

I agree  with  your  decision  points  for  the  counselor.  And  of  course,  this  is  assuming  that  John  disclosed  his 
condition  to  his  counselor.  I think  there  are  more  decision  points  that  Mary’s  counselor  has  to  make. 

Mary’s  counselor  now  has  the  responsibility  to  look  into  John’s  case  to  see  if  John  told  his  counselor  about 
his  condition.  If  he  did,  then  he  is  obligated  to  the  ethical  codes  H.2.a.,  H.2.b.,  H.2.d.,  and  H.2.e. 

Supporting  Example  2: 

I was  working  on  the  revision  and  I notice  something  I need  clarification  on.  In  my  answer,  I used  the 
example  of  consumer-oriented  as  my  Language  Arts  department  and  their  evaluation  of  curriculum 
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materials  for  an  adoption  next  year.  You  said  this  was  incorrect.  In  our  notes,  it  says  “evaluation  of 
curriculum  packages.”  How  is  this  different?  Please  clarify? 


Challenging  patterns  seemed  to  emerge  in  both  cases  when  controversial  topics  were  involved.  In  one  case, 
challenging  communication  was  actually  second  highest  in  frequency.  Overall,  the  challenging  pattern  was  lower 
than  initiating  or  supporting  patterns.  An  example  of  a comment  coded  in  the  challenging  category  is  listed  below. 

Challenging  Example: 

Let’s  say  the  counselor  did  know  John  was  in  a relationship  with  Mary,  Maria,  or  whoever.  I think 
Kitchener’s  principle  of  Nonmalficence  applies.  The  counselor  was  to  avoid  doing  harm.  He  did  not  harm 
John  but  in  the  long  run  he  did  harm  to  Mary  because  he  had  a duty  to  inform  her  of  the  danger  she  was  in. 
Also,  the  principal  of  truthfulness  because  informed  consent  is  part  of  that.  The  counselor  should  have  told 
John  that  there  were  limits  and  exceptions  to  confidentiality. 

Both  cases  exhibited  low  monitoring  patterns.  An  example  of  a comment  coded  as  monitoring  is  noted  below. 

Monitoring  Example: 

FYI,  In  case  you  haven’t  checked  the  Q&A  section.  Dr.  C.  said  we  need  to  make  sure  that  we  adapt  that 
checklist  on  p.  218  to  meet  the  needs  of  our  program  evaluation.  We  still  have  to  do  that,  right? 

Summarizing  communications  were  observed  in  only  one  of  the  cases  (Case  2).  An  example  is  identified 

below. 


Summarizing  Example: 

This  is  a great  example  of  problems  of  working  in  rural  communities  with  only  one  therapist  available.  I 
also  see  some  similarity  to  what  ministers  and  clergy  must  inevitably  deal  with:  multiple  relationships. 
What  I see  here  that  seems  most  striking  is  the  counselor’s  discussion  on  informing  the  parent  and 
discussing  this  duality.  But  she  did  not  make  the  same  attempt  with  Chris  who  is  the  CLIENT.  Hence,  I 
believe  he  was  treated  in  a second-class  manner  and  the  mother  was  given  more  power  in  the  counseling 
process  than,  again,  Chris  the  client.  Also,  there  is  the  divergence  of  obligations  due  to  the  counselor  again 
putting  the  parent’s  needs  before  the  child. 

Overall,  initiating  and  supporting  communication  patterns  dominated  the  online  discussions.  Challenging 
and  monitoring  patterns  were  exhibited  lower  overall.  However,  in  one  case,  these  patterns  came  out  higher  than 
initiating.  Summarizing  occurred  in  only  one  of  the  cases.  Table  1 provides  data  on  the  patterns  of  communication 
coded  using  the  identified  criteria. 

Table  1.  Patterns  of  Communication  Generated  by  Group/Case  Online  Discussion 


Case  One 

Activity  1 

Activity  2 

Activity  3 

Activity  4 

Total  Postings 

Initiating 

32 

22 

26 

83 

163 

Supporting 

38 

25 

29 

92 

194 

Challenging 

0 

0 

0 

22 

22 

Monitoring 

2 

0 

0 

0 

2 

Summarizing 

0 

0 

0 

0 

0 
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Case  Two 

Activity  1 

Activity  2 

Activity  3 

Activity  4 

Total  Postings 

Initiating 

10 

8 

5 

5 

28 

Supporting 

40 

16 

15 

13 

84 

Challenging 

22 

14 

10 

9 

55 

Monitoring 

0 

3 

5 

2 

10 

Summarizing 

1 

3 

1 

1 

6 

Interaction  Patterns 

Response  to  a posting  (RP)  patterns  of  interaction  (level  2)  were  high  in  both  cases.  Reply  to  a response 
(RR)  patterns  (level  3)  tended  to  be  lower  in  occurrence  compared  to  RP  patterns.  Only  in  Case  2 did  students  use 
level  4 replies.  Certain  topics  seemed  to  generate  higher  levels  of  interactivity,  perhaps  due  to  members’  orientation 
and  motivation.  Also,  time  to  complete  an  activity  or  discussion  affected  the  interaction  patterns  with  a positive 
relationship  between  level  of  interaction  and  time.  In  Case  2,  the  levels  of  replies  to  replies  were  further  developed 
than  in  Case  1.  Case  2 was  the  course  in  which  some  initial  training  occurred  for  students  in  how  to  use  the  online 
tools.  Data  are  summarized  in  Table  2. 

Table  2.  Patterns  of  Interaction  Generated  by  Group/Case  Online  Discussion 


Case  One 

Coding 

Activity  1 

Activity  2 

Activity  3 

Activity  4 

Total  Postings 

Initial  Posting 

IP 

32 

22 

26 

83 

163 

Response  to  a Post 

RP 

21 

15 

18 

50 

104 

Reply  to  a Response 

RR 

8 

8 

9 

20 

45 

Reply  to  a Reply 

RRl 

7 

2 

2 

12 

23 

Reply  to  a Reply 

RR2 

3 

0 

0 

7 

10 

Reply  to  a Reply 

RR3 

1 

0 

0 

3 

4 

Reply  to  a Reply 

RR4 

__ 

— 

— 

— 

Reply  to  a Reply 

RR5 

- 

— 

— 

— 

__ 

Reply  to  a Reply 

RR6 

— 

— 

— 

Reply  to  a Reply 

RR7 

__ 

— 

— 

— 

__ 

Reply  to  a Reply 

RR8 

- 

- 

- 

- 

- 

Case  Two 

Coding 

Activity  1 

Activity  2 

Activity  3 

Activity  4 

Total  Postings 

Initial  Posting 

IP 

10 

8 

5 

5 

28 

Response  to  a Post 

RP 

24 

11 

9 

4 

48 

Reply  to  a Response 

RR 

20 

9 

6 

3 

38 

Reply  to  a Reply 

RRl 

13 

7 

5 

2 

27 

Reply  to  a Reply 

RR2 

5 

3 

3 

3 

14 

Reply  to  a Reply 

RR3 

2 

1 

4 

3 

10 

Reply  to  a Reply 

RR4 

1 

1 

1 

2 

5 

Reply  to  a Reply 

RR5 

1 

2 

0 

1 

4 

Reply  to  a Reply 

RR6 

0 

2 

0 

1 

3 

Reply  to  a Reply 

RR7 

0 

1 

0 

3 

4 

Reply  to  a Reply 

RR8 

0 

0 

0 

1 

1 

Topics  of  Discussion  and  Use  of  E-board  Space 

In  addition  to  looking  at  the  data  by  communication  patterns  and  interaction  levels,  other  patterns  became 
apparent  during  the  analysis.  The  data  were  sorted  into  five  categories  related  to  the  topic  or  thread  of  the  discussion 
(administrative,  project,  feedback,  technology,  and  personal).  The  topic  categories  were  developed  as  the  analysis 
was  done  and  different  topic  areas  emerged. 

Administrative  threads  occurred  related  to  postings  that  students  needed  to  do  for  the  course  (requirements 
or  instructions).  In  Case  I there  were  many  administrative  postings  (154)  compared  to  almost  none  in  Case  2 (3). 
Project  topics  related  to  project  tasks  and  their  completion.  In  both  cases,  the  number  of  postings  related  to  projects 
was  high.  Feedback  was  related  to  responses  made  by  the  instructor  or  other  students  on  how  students  were  doing 
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with  completing  their  projects  or  with  administration  of  the  e-board.  Only  students  in  Case  1 posted  feedback 
comments.  Technology  comments  related  to  issues  with  the  e-board  technology  itself.  Fourteen  such  comments 
appeared  in  Case  1 and  none  in  Case  2.  This  might  be  attributed  to  initial  student  training  in  Case  2.  Personal  coding 
occurred  with  topics  not  related  to  the  course.  While  these  postings  were  minimal,  the  content  of  the  messages 
seemed  important  to  the  community  building  aspects  of  the  courses.  Table  3 summarizes  the  common  threads  in  the 
online  discussions. 

Four  common  patterns  of  use  of  space  were  observed  in  both  cases:  social  space,  communication  space, 
discussion  space,  and  information  sharing  space.  Social  space  was  used  when  students  used  the  e-board  to  share 
personal  messages  that  contributed  to  a sense  of  community.  This  occurred  in  both  cases,  but  was  minimal. 
Communication  uses  included  using  the  space  for  assignment  related  interactions,  but  not  content  related,  for 
example,  students  discussing  appropriate  roles  for  team  members  or  progress  on  individual  components  of  the 
assignment,  setting  up  meetings,  and  answering  questions  in  order  to  complete  the  assignment  on  time.  These  uses 
occurred  in  both  cases.  Discussion  use  was  coded  when  content  appropriate  interactions  occurred  directly  related  to 
the  course  assignment,  for  examp  le,  students  discussing  the  merits  of  various  points  in  the  case.  These  interactions 
facilitated  collaboration  and  assignment  completion.  These  were  the  primary  interactions  in  the  e-board  space. 
Information  sharing  use  included  providing  facts  and  data  for  team  members  and  sharing  written  products  for 
assembly  into  a team  product.  This  was  the  second  most  prevalent  use  of  the  e-board  space. 

Table  3.  Common  Thread  Across  Online  Discussions 


Case  One 

Discussion  1 

Discussion  2 

Discussion  3 

Discussion  4 

Administrative 

21 

8 

45 

80 

Project 

19 

27 

9 

46 

Feedback 

26 

4 

1 

26 

Technology 

6 

5 

— 

14 

Personal 

— 

3 

__ 

9 

Total  Postinas 

72 

47 

55 

175 

Case  2 

Discussion  1 

Discussion  2 

Discussion  3 

Discussion  4 

Administrative 

3 

Project 

76 

43 

33 

29 

Feedback 

— 

— 

- 

Technology 

— 

— 

Personal 

— 

— 

__ 

1 

Total  Postings 

76 

46 

33 
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Summary  of  Findings 

It  seems  that  the  lower  levels  of  communication  patterns  (initiating,  supporting)  were  most  evident  in  the 
online  discussions.  Students  may  need  to  learn  new  roles  and  new  communication  patterns  in  the  online  environment 
to  stimulate  the  higher-level  communication  patterns  (challenging,  summarizing,  monitoring).  Structuring  online 
activities  and  following  sound  instructional  design  principles  may  lead  to  more  synthesizing  (summarizing)  and 
challenging  communication  patterns. 

Initial  levels  of  response  or  interaction  patterns  were  also  more  evident  in  the  cases.  Students  were  most 
likely  to  respond  to  an  initial  posting  (level  2),  and  much  less  likely  to  reply  to  a response  (level  3)  and  not  likely  at 
all  to  reply  to  a reply  (level  4).  Thus  the  interactions  appear  much  less  like  a discussion,  where  conversation  builds 
upon  previous  responses,  and  more  like  a question  and  answer  scenario.  This  was  particularly  tme  in  Case  1. 

Perhaps  orienting  and  training  students  in  how  to  respond  in  the  asynchronous  e-board  may  lead  to  more  complex 
reply  and  response  patterns. 

Administrative  and  technology  comments  appeared  primarily  in  the  course  where  no  initial  student 
orientation  to  the  technology  occurred.  The  primary  topics  evident  related  to  the  projects,  which  was  appropriate. 
Communicating  about  the  project  guidelines  and  organizing  group  activities,  discussing  content,  and  sharing 
information  were  all  achieved  in  the  online  environment.  Social  comments  appeared  to  contribute  to  the  sense  of 
community  online. 
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Suggestions 

The  following  suggestions  may  lead  to  enhanced  communication  patterns  in  online  instructional 
environments.  An  increased  emphasis  on  orientation  and  training  of  students  to  operate  in  the  online  environment 
seems  warranted.  Students  should  be  provided  with  examples  of  “good”  discussions.  Roles  might  be  explicitly 
defined.  Ensuring  that  students  understand  the  technical  aspects  of  how  to  post  responses  may  increase  the  higher 
levels  of  responding.  Students  may  also  need  time  to  practice  using  the  technology. 

Providing  structure  seems  to  enhance  the  communication  patterns.  First,  instructors  should  select  topics  that 
lend  themselves  to  a discussion  format.  Students  need  specific  guidelines  of  engagement  that  will  help  them 
understand  the  expectations  for  the  discussion.  Students  should  be  provided  opportunities  for  both  group  interactions 
as  well  as  one-on-one  interactions.  Students  should  be  encouraged  to  become  reflective  practitioners  by  asking  them 
to  include  lessons  learned  from  their  experiences. 

Attending  to  instructional  design  principles  should  enhance  the  learning  environment.  Learners  should  be 
able  to  focus  on  key  components  of  what  they  are  learning.  They  must  be  able  to  connect  what  they  know  to  what 
they  are  learning.  They  must  be  active  participants  in  the  learning  process.  And  they  need  feedback  on  their  learning 
attempts.  Use  of  the  asynchronous  discussion  areas  seems  particularly  appropriate  for  enhancing  active  participation 
in  the  learning  environment. 


Insights  for  Faculty  Members 

As  faculty  members  integrate  technologies  such  as  asynchronous  discussion  tools  into  their  courses,  they 
may  need  to  adapt  to  new  roles.  They  must  serve  as  sponsors,  experts,  learners,  and  facilitators  (SELF).  They  must 
become  sponsors  of  the  technologies  they  attempt  to  integrate  into  their  courses.  Sponsors  express  positive 
enthusiasm  for  the  tools  used  to  support  student  learning  and  provide  opportunities  for  the  students  to  become 
oriented  to  the  technology.  Faculty  must  create  online  learning  spaces  to  provide  the  content  expertise.  Working 
with  instructional  designers  and  technical  support  staff,  the  faculty  expert  can  design  an  effective  learning 
environment.  Third,  the  faculty  member  must  be  open  to  learning  new  ways  to  use  technology  to  support  the 
teaching  and  learning  process.  And  finally,  in  this  new  learning  environment,  the  faculty  member  must  become  a 
facilitator,  guiding  the  discussions  and  assisting  students  in  acquiring  new  knowledge.  Teaching  in  an  online 
environment  requires  a faculty  member  to  engage  with  the  technology,  the  students,  the  content,  and  the 
instructional  design  components.  This  engagement  interacts  with  the  faculty  roles  of  sponsor,  expert,  learner,  and 
facilitator  and  creates  a complex  teaching  model  that  might  be  represented  as  in  Figure  2.  How  one  manages 
communication  so  that  it  enhances  learning  in  this  complex  model  is  a task  that  will  challenge  faculty  to  rethink  the 
instructional  process. 
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Abstract 

The  purpose  of  this  study  was  to  find  if  there  was  a predictive  relationship  between  a classroom  teacher’s 
comfort  and  proficiency  with  using  technology  (referred  to  as  attitude)  and  level  of  technology  implementation;  and 
if  there  was  a predictive  relationship  between  teachers'  beliefs  and  level  of  technology  implementation.  The 
instrument  used  in  this  study  was  the  Level  of  Technology  Implementation  (Moersch,  2001 ).  The  findings  show  that 
there  was  a moderately  positive  correlation  between  attitudes  and  technology  implementation  and  between  beliefs 
and  level  of  technology  implementation.  A multiple  regression  of  age,  years  of  teaching  experience,  attitudes  and 
beliefs  resulted  in  attitudes  and  beliefs  being  the  strongest  predictors  of  higher  levels  of  technology  implementation. 


Introduction 

Researchers  have  illustrated  how  computer  technologies  can  be  powerful  tools  that  can  greatly  impact  how 
and  what  students  learn  (David,  1994;  Dwyer  et  ah,  1991;  Dwyer,  1994;  Milken  Exchange,  1999).  Despite  the 
increase  of  availability  of  computers,  researchers  and  educators  still  report  that  integrating  technology  into 
classroom  practices  is  not  easily  accomplished  (Ertmer,  1999;  Ertmer  et  al,  1999;  Hadley  & Sheingold,  1993;  Jones, 
2001;  Schrum,  1999).  Researchers  have  identified  many  barriers  including  teachers’  preferred  instructional  methods 
and  their  corresponding  beliefs  about  teaching  and  learning  (Hannafin  & Savenye,  1993;  Niederhauser  & Stoddaart, 
2001;  Pierson,  2001;  OTA,  1995).  It  is  important,  therefore,  to  examine  teachers’  belief  systems  and 
institutionalized  instructional  routines  that  support  or  impede  technology  use,  in  order  to  overcome  barriers  to 
successful  technology  integration.  Technology  is  prevalent  in  our  schools,  however,  the  pressing  issue  currently 
centers  on  how  technology  is  being  used  or  not  being  used  in  the  teaching  and  learning  process  realizing  that  student 
achievement  may  be  compromised  by  poor  technology  integration  practices. 

There  is  little  doubt  that  technology  plays  an  important  role  in  our  schools,  considering  the  wide  range  of 
computer  applications  in  today’s  world.  Consequently,  teachers  must  be  prepared  to  use  computers  in  their 
instructional  practices  and  to  do  so  effectively.  Research  has  shown  that  many  elementary  teachers  still  feel 
uncomfortable  using  computers  (Guha,  2000;  Marcinkiewicz,  1993;  McDermott  & Murray,  2000  Rosen  & Weil, 
1995;  Scheffler  & Logan,  1999).  Rosen  & Weil  (1995)  investigated  technophobia  (computer  anxiety)  and 
discovered  that  between  one -third  and  two -thirds  of  teachers  in  their  study  were  not  using  computers  at  home  or 
with  their  students  because  of  a lack  of  confidence.  Similar  findings  were  reported  by  McDermott  & Murray 
(2000),  and  Guha  (2000).  In  both  studies  a positive  correlation  was  found  between  low  computer  use  and  low  levels 
of  comfort.  Schechter  (2000)  found  a strong  predictive  relationship  between  higher  levels  of  technology  use  and 
comfort,  confidence  and  proficiency  with  using  computers.  Attitudes  about  the  value  of  technology  use  in  education 
have  also  been  shown  to  affect  levels  of  technology  implementation,  for  examp  le  studies  have  shown  that  beliefs 
and  attitudes  about  the  advantages  of  teaching  with  technology  are  significantly  correlated  with  higher  levels  of 
technology  implementation  (Becker,  1994;  Lebrutto,  2001).  Access  is  becoming  less  of  a concern  (Becker,  1999). 

It  is  more  important  to  look  at  how  teachers  are  learning  to  implement  computers  and  how  they  feel  about 
technology  use  because  the  teacher  is  key  in  the  eventual  success  or  lack  of  success  of  any  computer-in-education 
initiative  (Collis,  1996 

The  purpose  of  this  study  is  to  examine  barriers  that  affect  teachers’  effective  technology  integration  in 
their  instructional  practices.  Two  barriers  are  the  primary  focus:  teachers’  beliefs  about  teaching  and  learning,  and 
teachers’  attitudes  towards  technology  use.  Practical  implications  of  the  results  of  this  study  may  provide  valuable 
insight  to  be  considered  when  planning  teacher  training  (inservice  and  preservice)  and  sustaining  technology  use  in 
meaningful  ways  that  enhance  the  teaching  and  learning  process. 
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Literature  Review 


Beliefs  About  Teaching  and  Learning 

Any  innovation  whether  in  education  or  in  the  workplace  creates  some  personal  dissonance.  It  is 
important,  therefore,  to  examine  how  teachers’  beliefs  support  or  inhibit  an  innovation  (i.e.  technology  integration) 
before  expecting  acceptance  and  change  to  occur.  Personal  belief  systems  exert  a powerful  influence  on  teachers’ 
decision-making  and  on  the  instructional  practices  they  use  in  their  classrooms  (Niederhauser,  D.  & Stoddart,  T., 
2001;  Pajares,  M.F.,  1992).  Instructional  changes  involving  innovations  (computers)  will  be  filtered  through 
teachers’  structure  of  knowledge  and  beliefs  about  teaching  and  learning.  Ertmer  (1999)  talks  about  teachers’  belief 
systems  in  terms  of  what  she  calls  second-order  barriers  that  interfere  with  change  in  teaching  practices.  She  states 
that  teachers  may  not  be  aware  of  their  own  underlying  beliefs  about  teaching  and  learning.  Thus,  it  is  important  to 
bring  about  an  awareness  of  what  these  beliefs  are  and  the  underlying  values  and  assumptions  that  are  the 
foundations  of  these  beliefs.  These  second  order  barriers  could  cause  more  difficulties  because  they  are  less  tangible 
than  first  order  barriers.  First  order  barriers,  according  to  Ertmer  (1999),  involve  a lack  of  access  to  hardware  and 
software,  as  well  as  a lack  of  training,  time,  and  support  for  technology  integration.  Ertmer  suggests  that  both  first- 
order  and  second-order  barriers  be  addressed  simultaneously  and  recursively  as  both  types  of  barriers  surface 
throughout  the  technology  integration  process. 

Teachers’  beliefs  are  conditioned  by  how  they  were  taught,  and  early  in  the  education  process,  teachers 
have  formed  a philosophy  of  teaching  and  learning  that  becomes  quite  ingrained  and  resistant  to  change  (Dwyer,  D., 
Ringstaff,  C.,  and  Sandholtz  J.,  1991;  McKinney,  M.,  Sexton,  T.,  and  Meyerson,  M.,  1999;  Norum,  K.,  Grabinger 
R.S.,  and  Duffield,  J.,  1999).  Teachers  must  question  these  beliefs  as  they  attempt  to  change  teaching  practices.  For 
example,  technology  integration  precipitates  changes  in  the  learning  environment  that  impacts  on  teachers’  and 
students’  roles  in  the  classroom.  It  might  take  some  adjusting  to  the  idea  that  teachers  can  learn  from  students  who 
may  know  more  about  technology  than  they  do.  This  new  learning  environment  may  cause  teachers’  a sense  of  loss 
of  control  since  the  one-way  transmission  of  knowledge  model  (teacher  ? student)  may  be  incompatible  with 
allowing  students  to  be  resources  in  the  classroom  (Ertmer,  P.,  et  ah,  1999).  Teachers  must  be  ready  to  reexamine 
their  teacher  roles.  Resistance  to  this  new  role  may  clearly  be  connected  to  teachers’  preference  and  adherence  to 
old  patterns  of  instructional  methods  (Hannafin,  R.D.  & Savenye,  W.C.,  1993;  Johnson,  1997),  and  their  belief  that 
responsibility  for  learning  rests  with  them.  If  teachers  believe  that  they  are  to  be  in  charge  at  all  times  and  the  sole 
dispensers  of  knowledge,  then  relinquishing  that  role  will  require  some  shifts  in  deeply  held  beliefs. 

Niederhauser  & Stoddart  (2001)  found  that  teachers  who  hold  more  traditional  beliefs  about  teaching  and 
learning  tend  to  use  more  didactic  instructional  methods  (i.e.  lecturing)  and  more  teacher-centered  practices,  while 
teachers  who  hold  more  constructivist  beliefs  about  learning  tend  to  use  more  student-centered,  inquiry  based 
methods.  Teachers’  beliefs  about  how  learners  learn  are  expressed  through  their  instructional  practices.  The 
implication  would  be  that  technology  integration,  then,  might  be  affected  by  teachers’  current  instructional  practices. 

Furthermore,  Niederhauser  & Stoddart  (2001)  examined  the  relationship  between  teachers’  instructional 
perspectives  and  beliefs  about  learning,  and  their  corresponding  use  of  educational  software.  They  classified 
instructional  software  into  two  categories  based  on  the  design  and  purpose  of  the  software:  skilled -based  software, 
which  embodied  a traditional  transmission  approach  to  instruction,  and  open-ended  software  that  embodied  a 
constructivist  learner-centered  approach.  The  authors  found  a consistent  relationship  between  teachers’  perspectives 
about  the  instructional  uses  of  computers  and  types  of  software  they  used  with  their  students.  Teachers  who  only 
used  open-ended  software  had  a strong  learner-centered  orientation  and  weak  computer-directed  orientation,  while 
teachers  who  used  only  skill-based  software  had  the  strongest  computer-directed  and  lowest  learner-centered 
orientations.  When  the  authors  looked  at  K-2  teachers  and  compared  them  to  grades  3-6  teachers,  results  indicated 
that  K-2  teachers  favored  the  use  of  skill-based  software  over  open-ended  software  to  a greater  degree  than  the  3-6 
grade  teachers.  Understandably,  this  could  be  due  to  the  fact  that  teachers  at  the  primary  grade  levels  need  to 
emphasize  foundational  skills  in  the  early  grades  and  the  fact  that  there  is  wide  availability  of  skill-based  software 
designed  for  young  children. 

Honey  & Moeller  (1990)  found  that  there  were  certain  discernible  patterns  between  teachers’  pedagogical 
beliefs,  their  instructional  practices  and  the  integration  of  technology  into  their  classroom  practices.  High-tech 
teachers  employed  more  progressive  educational  practices,  such  as  use  of  inquiry  and  discovery  skills,  project- 
oriented  work,  group-based  activities  and  hands-on  activities.  Dirksen  and  Tharp  (2000)  evaluated  a Goals  2000 
project  carried  out  in  a rural  western  school  district  where  participants  were  being  trained  to  develop  skills  with  a 
variety  of  software  applications  and  the  integration  of  technology  within  the  curriculum.  They  observed  that 
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teachers  who  had  their  students  create  presentation  products  using  the  internet  as  a research  tool,  viewed  computers 
as  generative  learning  tools  to  be  used  to  help  students  develop  higher-order  thinking  skills.  These  teachers  were 
more  skilled  in  using  the  internet  and  presentation  software  from  which,  the  researchers  concluded,  the  students 
benefited.  Those  teachers  who  used  computers  as  a word  processing  tool,  primarily  had  students  type  their  final 
products  such  as  reports,  stories,  and  poems.  With  teachers  possessing  less  skill  in  computer  use  and  less  knowledge 
of  a variety  of  software  applications,  students  were  using  the  computer  as  a workbook  and  using  drill  and  practice 
computer  programs  designed  to  reinforce  knowledge  and  skills  introduced  by  these  teachers. 

In  technology  rich  environments,  the  process  of  learning  is  transformed  to  permit  more  student 
participation  and  a focus  on  students’  individual  learning  styles,  according  to  a study  done  by  McDermott  & Murray 
(2000),  conducted  in  a K-2  building  in  a Midwest  middle  class  suburban  city.  These  authors  felt  that  teachers’ 
philosophies  about  how  learners  learn  must  be  re-evaluated  in  order  to  embrace  the  idea  that  learners  are 
constructors  of  knowledge  and  meaning.  In  a technology-rich  environment,  the  role  of  teacher  must  shift  to  focus 
more  on  the  students’  active  role  in  learning.  For  example,  the  Apple  Classroom  of  Tomorrow  (ACOT)  project 
exemplified  how  teachers’  beliefs  and  practices  were  changed  as  teachers  realized  the  benefits  of  the  role  shifts  not 
only  with  students,  but  also  with  other  teachers  with  whom  they  collaborated.  The  teachers  in  this  project  were  given 
access  to  multiple  computers  in  their  classrooms  and  laptops  for  personal  use.  Technology,  in  other  words,  was 
infused  into  their  teaching  and  learning  environment.  Teachers  in  the  ACOT  project  helped  one  another  across 
content  areas  and  team  taught  in  the  context  of  project-based  instruction  (Dwyer,  D.,  Ringstaff,  C.,  and  Sandholtz  J., 
1991;  Dwyer,  1994).  The  project  seems  to  hold  some  promise  in  changing  teachers’  beliefs  that,  in  turn,  might 
result  in  a change  in  teaching  practices  to  include  higher  levels  of  technology  integration. 

Teachers’  beliefs  about  the  efficacy  of  the  role  of  technology  in  education  also  may  have  an  influence  on 
how  computers  are  used  in  the  classroom.  Becker  (1994)  differentiated  exemplary  computer-using  teachers  from 
other  teachers  and  found  that  exemplary  teachers  used  technology  differently  and  that  their  goals  for  computer  use 
were  different.  Students  used  computers  in  “consequential”  activities  such  as  creating  newsletters,  producing  a 
school  newspaper  or  writing  for  the  school  yearbook  (i.e.  authentic  tasks).  There  was  a greater  use  of  software  for 
higher  order  thinking  activities  such  as  interpreting  data,  reasoning,  writing,  solving  real-world  problems,  and 
conducting  scientific  investigations.  Less  time  was  spent  on  recreational  activities  such  as  game  playing. 

In  a study  conducted  by  Ertmer  et  al  (1999),  there  appeared  to  be  a connection  between  teachers’  uses  of 
technology  and  their  beliefs  about  the  role  of  technology.  How  teachers  used  technology  was  related  to  what  role 
they  believed  technology  should  play  in  the  curriculum.  When  they  interviewed  teachers,  they  found  that  those 
teachers  who  did  not  value  technology  as  a tool  for  enhancing  learning,  used  computers  as  an  optional  activity. 
Students  were  permitted  to  use  computers  as  a reward  after  completing  “regular”  classroom  work.  These  teachers 
did  not  see  the  relevancy  of  computer  use  as  integral  to  the  curriculum.  They  felt  that  once  they  taught  new  skills, 
students  could  use  the  computers  for  drill  and  practice  work  to  reinforce  what  was  already  taught.  In  this  study, 
however,  one  teacher  did  come  to  change  her  beliefs  about  using  computers  in  a way  that  “drove”  her  curriculum. 
She  began  using  computers  not  only  as  a support  to  what  she  taught,  but  as  an  integral  part  of  her  teaching. 

Teachers’  Self-Beliefs  (Efficacy)  and  Attitudes  about  Technology  Use 

Teachers  beliefs  in  their  ability  to  positively  affect  student  learning  impacts  on  their  willingness  to  embrace 
innovative  teaching  approaches  according  to  McKinney  et  al.  (1999).  Teachers  must  believe  that  an  innovation  will 
enhance  student  learning  before  they  will  be  receptive  to  change  and  there  is  evidence  that  most  teachers  do  see  the 
advantages  of  technology  integration  into  the  curriculum  in  terms  of  enhanced  student  learning  and  increased 
motivation  (Ertmer,  P.,  1999;  Ertmer,  P.  et  al.,  1999).  But,  teachers  have  to  believe  that  they  are  capable  of 
teaching  effectively  with  technology.  Honey  and  Moeller  (1990)  have  shown  that  teachers  who  successfully 
integrated  computers  into  their  teaching  regarded  themselves  as  learners  along  with  their  students  and  were  highly 
motivated  because  they  expended  their  own  time  and  effort  in  learning  to  use  computers.  These  teachers  exhibited 
confidence  in  their  ability  to  learn  new  technology  skills.  Similar  findings  were  noted  in  studies  conducted  by 
Sheingold  and  Hadley  (1990),  Marcinkiewicz  (1994),  Lumpe  and  Chambers  (2001)  and  Beaudin  (1999).  Lumpe  & 
Chambers  (2001)  found  that  self-efficacy  beliefs  (confidence)  were  a significant  predictor  of  teachers’  self-reported 
use  of  technology-related  engaged  learning  practices.  Similarly,  Schechter  (200)  found  a significant  correlation 
between  a teacher’s  comfort  and  proficiency  with  using  computers  and  the  degree  to  which  he/she  employed 
computers  a a process,  product  and  tool  to  solve  authentic  curriculumrelated  problems.  In  other  words,  teachers 
with  higher  levels  of  comfort  and  proficiency  tended  to  implement  technology  at  higher  levels  of  integration.  Lack 
of  confidence  and/or  comfort  can  be  powerful  de-motivators  to  technology  implementation  and  integration.  Some 
teachers  may  be  positively  predisposed  to  using  technology,  but  still  exhibit  a great  deal  of  technophobia  due  to  bad 
prior  experiences  with  technology,  or,  they  may  simply  have  a great  fear  of  failure  (Rosen,  L.  and  Weil,  M.,  1995). 
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It  is  important  that  teachers  have  the  opportunity  to  rethink  and  analyze  their  beliefs  and  values  in  relationship  to 
technology  integration  then  confront  their  fear  of  technology.  They  must  examine  their  current  teaching  practices 
and  be  prepared  to  adapt  to  the  changes  that  technology  brings  to  the  classroom. 

Clearly  changing  beliefs  takes  time  and  as  some  researchers  have  found,  it  seems  to  follow  a 
developmental  pattern  (Johnson,  1997;  Pajares,  1992).  Pajares  (1992)  talks  about  conceptual/belief  change  in  terms 
of  Piaget’s  two  processes  of  assimilation  and  accommodation.  Assimilation  is  the  process  whereby  new  information 
is  incorporated  into  existing  beliefs  and  accommodation  takes  place  when  new  information  cannot  be  assimilated 
and  existing  beliefs  must  be  replaced  or  reorganized.  In  light  of  innovations  and  technology  use,  for  teachers  to 
become  adopters  of  technology  use  instead  of  resisters,  fundamental  changes  to  their  belief  system  may  be  required 
before  new  ideas  become  incorporated  into  a transformed  belief  system.  McKinney  et  al.  (1999)  talked  about  the 
change  process,  which  they  believed  occurred  in  a sequential  manner.  They  believed  that  there  were  three  stages  of 
innovation:  initiation,  implementation,  and  refinement.  The  authors’  theoretical  model  suggests  that  participants 
involved  in  an  innovation  move  through  these  stages  expressing  different  sets  of  concerns.  The  stages  of  concern 
coincide  with  the  stages  of  innovation  ranging  from  little  concern  or  involvement  with  the  innovation  at  the  outset. 
With  more  involvement,  concerns  seem  to  be  centered  on  desires  to  know  more  about  the  innovation  along  with  the 
availability  of  training  or  support  for  the  innovation.  This,  in  turn,  leads  to  concerns  about  practices  involved  with 
implementation,  then  finally,  to  refinement  that  involves  finding  better  ways  to  incorporate  the  innovation. 
Throughout  these  stages,  self-efficacy  is  affected.  In  the  beginning  of  an  innovation,  certain  sets  of  concerns  tend  to 
be  expressed  by  participants  with  lower  efficacy  beliefs  while  those  with  higher  efficacy  have  concerns 
characteristic  of  the  later  stages  of  change,  that  is,  those  of  impact.  Findings  from  a study  conducted  by  Mills  (1999) 
support  the  position  that  teachers’  concerns  and  perceptions  of  an  innovation  influence  the  way  in  which  they 
implement  technology.  Teachers’  concerns  and  perceptions  must  be  considered  if  schools  are  to  experience 
significant  change  or  reform  with  respect  to  integration  of  computer  technology. 

Understanding  the  process  of  change  and  feelings  (e.g.  comfort  and  confidence)  about  change  in 
educational  settings  can  help  educational  reformers,  policymakers,  and  administrators  design  better  training 
programs  to  bring  about  positive  change  in  teacher  practices. 


Research  Questions 

1.  Is  there  a predictive  relationship  between  a classroom  teacher’ s comfort  and  proficiency  with  using  computers 
and  the  level  of  technology  implementation? 

2.  Is  there  a predictive  relationship  between  teachers’  beliefs  as  reflected  in  their  current  instructional  practices 
and  level  of  technology  implementation? 

Hypotheses 

Hypothesis  1:  There  is  a positive  correlation  between  a classroom  teacher’s  comfort  and  proficiency  with 
using  computers  (PCU)  and  the  level  of  technology  implementation. 

Hypothesis  2:  There  is  a positive  correlation  between  the  degree  to  which  a classroom  teacher  employs 
student-centered,  authentic,  problem-solving  strategies  (CIP)  and  level  of  technology  implementation. 

Definitions 

1.  Beliefs  about  teaching  and  learning  refer  to  teachers’  pedagogical  beliefs  about  how  children  learn.  These 
beliefs  have  an  epistemological  component  in  that  teachers’  beliefs  about  the  nature  of  reality  form  the 
basis  of  their  orientation  to  an  objectivist  or  constructivist  perspective.  Depending  on  this  orientation, 
knowledge  is  viewed  either  as  something  constructed  by  learners  or  transferred  intact.  This  has 
implications  for  the  learning  process,  which  can  be  viewed  as  an  active,  creative  and  socially  interactive 
one,  or  a passive  process  where  learners  receive  knowledge  into  supposedly  “empty  vessels’’. 

2.  Attitudes  are  defined  as  teachers’  feelings  of  comfort,  confidence  and  proficiency  in  using  technology. 

3.  Teachers’  beliefs  about  teaching  and  learning  are  measured  by  the  Current  Instructional  Practices  (CIP) 
measure.  The  CIP  profile  shows  an  inclination  toward  instructional  practices  consistent  with  learner- 
centered  approaches  and  indicators  of  engaged  learning  (Means,  1994).  These  indicators  are  demonstrated 
in  the  following  ways: 

a.  Children  are  engaged  in  authentic,  real-world,  problem-solving  tasks 

b.  Children  are  actively  involved  in  learning  activities 

c.  Children  work  collaboratively 

d.  Teacher  is  a facilitator  in  learning 

e.  Children  learn  through  exploration 
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4.  Teachers’  attitudes  are  measured  by  the  Personal  Computer  Use  Profile  (PCU),  which  indicates  a classroom 
teacher’s  comfort  and  proficiency  levels  with  using  computers. 

5.  Level  of  Technology  Implementation  (LoTi)  is  a measure  of  computer  use  and  implementation  by  a 
classroom  teacher.  It  consists  of  eight  discrete  implementation  levels  ranging  from  Non-Use  (Level  (0)  to 
Refinement  (Level  6).  As  a teacher  progresses  from  one  level  to  the  next,  a series  of  changes  to  the 
instructional  curriculum  is  observed.  The  instructional  focus  shifts  to  more  student-centered  practices. 
Technology  implementation  is  used  interchangeably  with  technology  integration. 

6.  Level  of  Technology  Implementation  (LoTi)  questionnaire  contains  50  items  developed  by  Moersch  (1994) 
that  measure  three  constructs:  level  of  technology  implementation,  level  of  comp  uter  use,  and  level  of 
current  instructional  practice. 

Limitations  of  Study 

This  study  is  looking  at  primarily  two  variables  affecting  level  of  technology  implementation,  attitudes  and 
beliefs.  There  are  other  variables  that  affect  technology  implementation  besides  beliefs  and  attitudes  or  several  of 
the  demographic  variables  selected  in  this  study.  However,  this  researcher,  in  examining  the  literature  on  factors 
influencing  computer  use,  has  concluded  that  teachers’  beliefs  and  attitudes  (i.e.  internal  barriers)  seem  to  be  the 
best  predictors  of  technology  implementation. 

In  addition,  this  study  is  examining  one  school  district  in  Central  Pennsylvania  and  elementary  teachers  in 
K-5.  Broad  generalizations  beyond  this  population  cannot  be  made. 

Methodology 

Participants 

Questionnaires  were  sent  to  84  elementary  teachers  (K-5)  in  four  elementary  schools  in  a Central 
Pennsylvania  School  District.  All  elementary  schools  were  in  a suburban  area,  which  was  culturally  homogeneous. 
Forty  questionnaires  were  returned  representing  a response  rate  of  48%.  85%  were  female  and  15%  were  male.  A 
majority  (57.5%)  of  the  teachers  had  more  than  20  years  of  teaching  experience  and  the  majority  (68%)  of  the 
teachers  were  in  the  40-59  age  group. 

Instrumentation 

Instrumentation  consists  of  the  Level  of  Technology  Implementation  (LoTi)  questionnaire.  The  LoTi 
questionnaire  was  developed  by  Christopher  Moersch  and  Learning  Quest,  Inc.  It  measures  teachers’  levels  of 
technology  use  at  the  instructional  curriculum  level,  not  their  proficiency  levels  in  technology  skills.  This 
questionnaire  was  selected  because  not  only  does  it  measure  level  of  technology  implementation,  it  also  builds  into 
the  instrument  the  Personal  Computer  Use  (PCU)  subscale  which  measures  teachers’  comfort  and  proficiency  using 
computers  referred  in  this  study  as  attitude.  LoTi  consists  of  a beliefs  subscale  (CIP),  which  measures  teachers’ 
beliefs  about  teaching  and  learning  as  evidenced  by  their  current  instructional  practices.  The  purpose  of  the 
instmment  is  to  be  used  to  plan  and  implement  staff  development.  The  LoTi  questionnaire  consists  of  50  questions 
(Moersch,  2001). 

Internal  reliability  of  the  questionnaire  was  studied  by  Schechter  (2000)  using  Cronbach’s  alpha.  The 
investigator  separated  the  instrument  into  its  three  component  parts  (LoTi,  PCU,  and  CIP),  and  found  fairly  high 
reliabilities  (r  = .7427,  .8148,  and  .7353,  respectively).  In  addition,  individual  reliability  estimates  of  individual 
levels  of  the  LoTi  demonstrated  high  levels  of  internal  consistency  on  levels  3 (r  = -I-.7037),  4B  (r  = -I-.7882),  6 (r  = 
-I-.8235),  PCU  (r  = -I-.8148),  and  CIP  (r  = -I-.7535).  The  instrument,  thus,  seems  to  possess  a measure  of  internal 
consistency  across  it  parts. 

Validity  has  not  been  addressed  by  Moersch,  nor  has  the  instrument  been  validated  in  research  to  date. 
However,  the  instrument  seems  to  possess  face  validity  and  content  validity  based  on  an  examination  of  the 
questionnaire  items.  For  example,  test  items  measuring  attitude  contained  words  such  as  comfort,  confidence  or 
referred  to  the  respondents’  proficiency  in  using  technology: 

“I  have  the  background  and  confidence  to  show  others  how  to  merge  technology  with  relevant  and  challenging 
learning  experiences  that  emphasize  higher-order  thinking  skills  and  student  relevancy  to  the  real  world. 

Items  having  to  do  with  beliefs  referred  to  teachers’  practices  evidencing  learner-centeredness: 

“I  use  my  students’  interests,  experiences,  and  desire  to  solve  authentic  and  relevant  problems  when  planning  a 
variety  of  computer-related  activities  in  my  classroom.” 

An  addendum  to  the  questionnaire  asked  for  demographic  information  such  as  age,  gender,  years  of 
teaching  experience,  and  grade  level  taught. 
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Construct  Validity 

Construct  validity  was  assessed  by  having  three  peers  evaluate  the  instrument  by  determining  which  items 
measured  beliefs,  attitudes,  and  technology  implementation.  This  researcher  identified  6 items  that  measured 
attitudes,  5 that  measured  beliefs  and  the  other  39  items  of  the  total  50  measured  technology  implementation.  There 
was  little  agreement  on  the  items  for  beliefs  and  attitudes  subscales.  Therefore,  validity  was  not  established. 

Data  Analysis 

Pearson  Product  Moment  correlations  were  computed  to  address  the  relationships  between  level  of 
technology  implementation  and,  comfort  and  proficiency  of  technology  use  (PCU),  and  beliefs  about  teaching  and 
learning  as  measured  by  Current  Instructional  Practices  (CIP).  Pearson  Product  Moment  correlations  were  also 
computed  for  two  of  the  demographic  variables,  age  and  years  of  teaching  experience.  An  independent  t-test  was 
performed  to  check  for  gender  differences  on  level  of  technology  implementation.  A one-way  ANOVA  was 
conducted  to  analyze  differences  among  teachers  who  taught  at  different  grade  levels  and  level  of  technology 
integration.  Finally,  a multiple  regression  analyzed  attitude,  beliefs,  age  and  years  of  teaching  experience  in  relation 
to  level  of  technology  implementation  to  establish  the  strongest  predictor  variables  for  higher  levels  of  technology 
implementation. 

Results 

Research  Question  1:  Is  there  a predictive  relationship  between  a classroom  teacher’s  comfort  and 
proficiency  (attitude)  with  using  computers  and  the  level  of  technology  implementation? 

There  was  a moderate  positive  correlation  (r=.540)  between  attitude  and  level  of  technology 
implementation  which  was  significant  at  the  .01  level  of  significance  indicating  that  the  more  teachers  felt 
comfortable  using  technology  and  the  more  confidence  they  expressed  in  using  technology  as  evidenced  by  their 
self-reported  proficiency,  the  higher  the  level  of  technology  implementation.  Teachers’  attitudes  are  a good 
predictor  of  higher  levels  of  technology  implementation. 

Research  Question  2:  Is  there  a predictive  relationship  between  teachers’  beliefs  as  reflected  in  their 
current  instructional  practices  ( CIP)  and  level  of  technology  implementation? 

A moderate  positive  correlation  (r=.524)  was  found  between  beliefs  and  the  level  of  technology 
implementation,  significant  at  the  .01  level  of  significance  indicating  that  the  more  teachers  were  inclined  toward 
learner-centered  practices  and  constructivist  principles,  the  higher  the  level  of  technology  implementation. 

Teachers’  beliefs  are  a good  predictor  of  higher  levels  of  technology  implementation. 

No  correlations  was  found  between  beliefs  and  attitudes  (r=.276)  as  expected  since  both  variables  measure 
different  constructs. 

Age  and  level  of  technology  implementation  showed  a slight  negative  correlation  (r=-.017)  indicating  that 
older  teachers  tended  to  be  at  lower  levels  of  implementation.  When  years  of  teaching  experience  was  correlated 
with  technology  implementation,  a slight  negative  correlation  (r=-.053)  was  found  indicating  that  teachers  with  more 
years  of  experience  tended  to  be  at  lower  levels  of  technology  implementation. 

No  significant  difference  was  found  between  male  female  teachers  (t=.  124,  p=.902)  with  respect  to  level  of 
technology  implementation.  With  respect  to  grade  level,  the  analysis  of  variance  revealed  significant  differences 
among  the  grade  levels  (f=5.927),  significant  at  the  .01  level  of  significance  (Table  1).  These  results  show  that 
teachers  in  various  grade  level  categories  (i.e.  K-2;  3-5;  K-5)  differed  significantly  on  levels  of  technology 
implementation. 


Table  I.  Comparison  of  Levels  of  Implementation  Scores  for  3 grade  Level  Categories 


Grade  Level 
Group 

Sum  of 
Squares 

df 

Mean  Square 

F 

Sig. 

Between  Groups 

6.118.242 

2 

3059.121 

5.927 

.006** 

Within  Groups 

19098.158 

37 

516.166 

Total 

25216.400 

39 

**  Significant  for  between  groups  at  <.001  level  of  significance 


A post  hoc  test,  Scheffe,  was  conducted  in  order  to  determine  where  the  significant  differences  occurred. 
Table  2 shows  significant  differences  between  group  2 (Grades  3-5)  and  group  3 (K-5),  and  group  1 (K-2)  and  group 
3 (K-5),  significant  at  the  .05  level  of  significance. 
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Table  2.  Schejfe  Post  Hoc  Test  on  AN OVA  for  Level  of  Technology  Implementation 


(I)GRLEVEL  (J)GRLEVEL 

Mean  Diff 
(I-J) 

Std.  Error 

Sig. 

1.0  2.00 

-23.9722* 

7.84923 

.016 

3.00 

1.5833 

10.93603 

.990 

2.0  1.00 

23.9722* 

7.84923 

3.00 

25.5556 

10.76906 

3.0  1.00 

-1.5833 

10.93603 

.990 

2.00 

-25.556 

10.76906 

.073 

*The  mean  difference  is  significant  at  the  .05  level. 


Table  3.  Means  of  Grade  Level  Groups  on  Levels  of  Technology  Implementation 


GRLEVEL 

Mean 

N 

Std.  Deviation 

1.00 

86.7500 

16 

19.09450 

2.00 

110.7222 

18 

25.84298 

85.1667 

6 

22.30172 

1 Total  1 

97.3000 

40 

25.42783 

Table  3 compares  the  means  of  each  group  and  shows  that  the  grade  level  category  2 (3-5)  had  a much 
higher  mean  than  the  other  two  groups  indicating  that  the  teachers  at  this  grade  category  tended  to  implement 
technology  to  a greater  extent  than  the  other  two  groups. 

Table  4 displays  the  results  of  the  stepwise  multiple  regression  for  beliefs,  attitudes,  age,  and  years  of 
teaching  experience  on  levels  of  technology  implementation.  Only  the  significant  (p<.05)  predictors  of  higher  levels 
of  technology  implementation  is  displayed.  Attitude  was  the  best  predictor  variable  (f=15.642,  p=<.000)  accounting 
for  approximately  30%  of  the  variance.  When  the  variable  of  beliefs  was  added  to  the  equation,  almost  45%  of  the 
variance  was  explained.  Both  variables,  in  combination,  seem  to  be  good  predictors  of  teachers  implementing 
technology  at  higher  levels. 


Table  4.  Regression  Results:  Significant  predictors  of  levels  of  technology  implementation 


Model 

R 

R Square 

Adj.  R Square 

Std.  Error  of 
The  Estimate 

1.  Attitudes 

.540 

.292 

.273 

21.68153 

2.  Belie  fs 

.666 

.444 

.414 

19.46916 

Discussion 

This  study  adds  to  the  research  on  factors  influencing  technology  use.  It  shows  that  beliefs  and  attitudes  as 
defined  in  this  study  are  good  predictors  of  increased  technology  implementation.  Teachers  that  are  comfortable  and 
confident  and  more  proficient  in  using  technology  tended  to  implement  technology  in  their  instructional  practices  to 
a higher  degree  than  those  who  were  less  comfortable,  confident  and  proficient.  It  also  indicated  that  teachers  who 
believe  that  learners  are  active  constructors  of  knowledge,  who  are  more  student -centered  in  their  instructional 
practices  and  view  themselves  as  facilitators  in  the  learning  process,  tended  to  implement  technology  to  a greater 
extent.  Demographic  factors  such  as  age,  gender  and  years  of  teaching  experience  did  not  have  a significant 
influence  on  technology  implementation.  However  with  respect  to  grade  level  categories,  teachers  teaching  grades 
3-5  showed  a significantly  higher  level  of  imp  lementation  than  the  other  two  groups  (i.e.  K-2,  K-5).  This  result 
indicates  that  teachers  at  the  higher  elementary  grades  tended  to  use  technology  to  a greater  extent  with  their 
students  than  the  others.  Further  investigation  of  the  K-5  group  might  reveal  that  most  of  these  teachers  deal  with 
K-3  students  and  fewer  students  at  the  grades  4-5  level,  thus  making  this  group  more  similar  in  composition  to  the 
level  1 group.  Most  of  these  teachers  taught  special  education  or  ESL.  It  may  be  reasonable  to  assume  that  these 
teachers  are  concerned  with  basic  skills  much  like  the  K-2  teachers.  More  analysis  would  be  required  to  support 
such  assumptions. 

This  study  was  limited  by  the  sample  size,  therefore,  these  findings  cannot  be  generalized  beyond  the  scope 
of  this  particular  school  district.  Further  research  comparing  school  districts  of  similar  characteristics  might  yield 
interesting  corroborative  data.  Comparisons  of  dissimilar  characteristics  would  also  point  to  different  variables 
having  more  power  to  predict  given  different  contexts  and  composition  of  demographics. 
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Moreover,  additional  research  conducted  on  teachers’  attitudes  could  explore  other  factors  within  attitudes 
that  might  prove  to  be  a stronger  predictor  of  technology  implementation.  For  exa  mple,  the  definition  of  attitudes 
could  include  how  teachers  feel  about  the  importance  of  technology  as  a tool  in  real-world  tasks,  and  how  teachers 
feel  about  the  positive  (or  negative)  effects  of  technology  use  on  student  achievement.  Factors  within  the  attitude 
construct  could  be  explored  to  find  an  optimum  composition  for  predicting  technology  implementation. 

If  attitudes  are  important  factors  in  influencing  technology  use,  then  finding  factors  that  influence  attitudes 
could  also  be  useful  for  the  purposes  of  creating  the  optimum  environment  to  promote  attitudes  conducive  to  higher 
levels  of  technology  implementation.  Influencing  factors  such  as  prior  learning  experiences  (how  teachers  were 
taught),  or  external  factors  such  as  school  climate,  adminis  trative  leadership,  availability  of  resources,  etc.  should  be 
investigated  to  gauge  their  influence  on  attitudes  and  beliefs.  For  example,  Ross,  Hogaboam-Gray  and  Hannay 
(1999)  found  that  confidence  to  implement  computer-based  instruction  was  influenced  by  increased  access  to 
technology  (resulting  in  teachers  using  technology  more),  administrative  support,  and  stronger  beliefs  about  the 
value  of  technology  in  the  classroom.  Marcinkiewicz  (1994)  found  that  teachers’  innovativeness,  defined  as  one’s 
willingness  to  change,  was  an  important  predictor  of  level  of  computer  use. 

In  this  study  beliefs  referred  to  pedagogical  beliefs.  Beliefs  comprise  a broader  constellation  of  constructs 
than  that  defined  in  this  study.  Beliefs  can  include  other  factors  missing  from  the  predictive  model  presented  here, 
such  as  beliefs  about  one’s  ability  to  implement  technology  (self-efficacy).  Self-efficacy  influences  how  people 
feel,  think,  and  motivate  themselves  to  behave,  and  refers  to  beliefs  in  one’s  capabilities  to  execute  actions  necessary 
to  manage  particular  situations  (Bandura,  1993).  Moreover,  teachers  may  have  beliefs  about  the  ability  of  external 
factors  or  people  to  enable  a person  to  reach  a goal  (e.g.  use  technology  successfully),  which  may  influence  whether 
or  not  a goal  is  attained.  Also,  the  belief  that  the  likelihood  that  a goal  can  be  attained,  given  a teacher’ s 
environment/teaching  context,  may  influence  achievement  of  that  goal  (Lumpe  & Chambers,  2001). 

Therefore,  a more  detailed  predictive  model  can  be  created  taking  into  consideration  the  multiple  aspects  of 
beliefs  and  attitudes  and  the  influences  that  affect  these  aspects  of  beliefs  and  attitudes.  Considering  many  aspects 
of  attitudes  and  beliefs  and  the  influences  affecting  these  variables  can  inform  better  professional  development 
models  which  is  the  impetus  of  this  type  of  research. 
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Abstract 

The  Web-Based  Training  Evaluation  Questionnaire  was  developed  in  order  to  evaluate  a new  Web-based 
instructional  product.  Currently,  few  instruments  are  available  for  the  evaluation  of  distance  learning  modules  that 
have  established  psychometric  properties.  This  paper  describes  the  study  that  resulted  in  the  successful  development 
of  the  instrument. 

Introduction 

The  number  of  courses,  certifications,  and  training  opportunities  offered  on  the  Internet  has  increased 
dramatically  over  the  last  few  years.  Distance  education  on  the  World  Wide  Web  is  being  increasingly  implemented 
in  the  workplace  (Lau,  2000).  The  number  of  companies  “using  the  Internet  for  training  purposes  grew  from  3%  in 
1996  to  38%  in  1999“  (American  Society  for  Training  and  Development,  2001,  p.  10).  The  International  Data 
Corporation  estimates  the  market  will  reach  $11.5  billion  in  2003  (Cohen,  2001). 

Even  though  many  online  courses  are  offered  for  education  or  training  purposes,  few  instruments  are 
currently  in  use  which  have  the  proper  established  psychometric  properties  to  evaluate  modules  in  the  Web-based 
environment.  The  Web-Based  Training  Evaluation  Questionnaire  (WBTEQ)  was  developed  to  assist  in  the 
evaluation  of  a new  instructional  product  that  was  designed  for  training  future  school  administrators  in  the  online 
environment.  The  researcher  hypothesized  the  instrument  would  yield  a high  internal  consistency  coefficient. 

Method 

Sample 

A pilot  study  was  conducted  to  establish  the  validity  of  the  instrument.  Eive  graduate  students,  who  were 
instructional  technology  or  communication  majors  and  who  were  enrolled  in  an  advanced  instructional  technology 
course  in  the  spring  semester  of  2002,  volunteered  to  participate  in  this  pilot  study.  These  students  were  instructed  to 
review  the  initial  version  of  the  instrument  and  examine  each  question  for  its  clarity  and  validity.  The  instrument 
was  administered  to  53  graduate  students  enrolled  in  three  instructional  technology  courses  taught  in  the  classroom 
environment  in  order  to  establish  the  instrument’s  reliability.  Then,  a follow-up  study  was  conducted  by 
administering  the  questionnaire  to  25  individuals  enrolled  in  a Web-based  module  pertaining  to  human  resources 
procedures  and  employment  contracts. 

Procedures 

Eor  the  pilot  study,  the  researcher  selected  students  in  the  advanced  instructional  technology  course  because 
these  individuals  had  expertise  in  the  online  distance  education  environment.  Eour  of  the  five  students  were 
instructional  technology  majors.  The  fifth  student  majored  in  communication  and  was  an  experienced  online  course 
designer. 

The  researcher  chose  to  administer  the  survey  to  53  graduate  students  in  face-to-face  instructional 
technology  courses.  After  the  first  data  collection  phase,  the  internal  consistency  reliability  was  determined  using  the 
Cronbach  alpha  coefficient  for  the  questionnaire  and  its  eight  subscales.  In  the  follow-up  study,  the  researcher 
administered  the  questionnaire  to  a sample  of  25  teachers  and  school  administrators  after  they  completed  a new 
Web-based  training  module;  24  participants  completed  the  questionnaire.  Once  again,  the  Cronbach  alpha 
coefficient  was  determined  for  the  total  scale  and  for  each  of  its  subscales. 

Instrumentation 

The  WBTEQ  has  49  four-point  Likert  Scale  items  and  5 open-ended  questions.  Items  on  the  scale  range 
from  1 to  4:  1,  strongly  disagree',  2,  disagree;  3,  agree;  and  4,  strongly  agree.  The  five  open-ended  questions 
address  trainees’  likes  and  dislikes  about  the  modules,  advantages  and  disadvantages  experienced  in  the  online 
environment,  and  suggestions  or  recommendations  for  improving  the  module. 

Commonly  used  Likert -type  scales  have  five  or  seven  response  sets  (Crocker  & Algina,  1986;  Gall,  Borg, 

& Gall,  1996;  Kerlinger  & Lee,  2000).  Kemp,  Morrison,  and  Ross  (1998)  advise  limiting  the  number  of  possible 
responses  on  scales  to  five  responses.  Qthers,  however,  suggest  using  a 4-point  scale  in  order  for  participants  to 
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avoid  selecting  neutral  responses.  This  practice  eliminates  bias  and  increases  the  instrument’s  reliability.  The  4-point 
scale  forces  participants  to  select  a statement  which  reflects  the  direction  in  which  they  lean  and  enables  researchers 
to  measure  the  strengths  of  their  opinions  and  attitudes  (Cronbach,  1946;  Pearson  & Carey,  1995).  Some  researchers 
oppose  this  method;  however,  Mercer  and  Durham  (2001)  found  there  was  no  statistically  significant  difference 
between  the  response  style  of  two  groups  which  were  given  a scale  with  and  without  a neutral  response  option. 
Issues  addressed  on  the  instrument  were  based  on  literature  in  the  field  pertaining  to  the  evaluation  of  online 
distance  education  courses.  These  issues  related  to  access  (Belanger  & Jordan,  2000;  Lau,  2000),  communication 
between  parties  (Bastiaens  & Martens,  2000;  Mantyla  & Gividen,  1997),  instructional  content  (Biner,  1993;  Dean  & 
Ripley,  1998;  Kemp  et  ah,  1998;  Khan,  1997;  Mantyla  & Gividen,  1997;  Schlough  & Bhuripanyo,  1998), 
instructional  design  (Aggarwal,  2000;  Belanger  & Jordan,  2000;  Berry,  2000;  Harrison,  1999;  Schlough  & 
Bhuripanyo,  1998),  administrative  and  technical  support  (Kemp  et  ah;  Khan,  1997;  Mantyla  & Gividen,  1997), 
equipment  and  technology  (Kemp  et  al.;  Khan,  1997;  Lau,  2000;  Mantyla  & Gividen,  1997),  learning  outcomes 
(Kirkpatrick,  1996;  Mantyla  & Gividen,  1997),  and  general  issues  such  as  learner  satisfaction,  reactions, 
experiences,  and  so  forth  (Khan,  1997;  Knowles,  1989;  Mantyla  & Gividen,  1997;  White,  1999;  Wideman  & 
Owston,  1999).  The  revised  questionnaire  had  eight  subscales:  (a)  access,  (b)  communication,  (c)  content,  (d) 
design,  (e)  support,  (f)  technology,  (g)  outcomes,  and  (h)  overall. 


Data  Analysis 

Eight  questions  of  the  49  Likert -type  scale  questions  on  the  questionnaire  were  negative  statements  and 
were  recoded  prior  to  the  analysis.  One  case  was  deleted  from  the  data  collected  during  the  pilot  study  because  more 
than  one-third  of  the  data  was  missing.  The  other  missing  data  were  substituted  with  the  series  mean.  The  data  were 
then  analyzed.  In  the  follow-up  study,  missing  data  were  substituted  with  the  mean,  and  the  data  were  examined  for 
univariate  outliers.  Statistical  assumptions  were  examined  in  order  to  avoid  a violation  of  the  assumptions. 

Results 

Initially,  the  instrument  had  55  items  and  5 open-ended  questions.  After  the  pilot  study,  the  instrument  was 
revised  to  adapt  the  recommendations  made  by  the  participants.  Five  items  on  the  instrument  were  identified  as 
redundant.  For  example,  the  instrument  had  questions  relating  to  the  attractiveness  of  the  screen  design;  however, 
participants  felt  it  was  unnecessary  to  ask  trainees  if  the  background  color  and  font  color  was  appealing  as  well. 

One  question  pertaining  to  graphics  having  text -based  descriptions  was  deleted  because  we  assumed  many  school 
administrators  had  no  previous  experience  with  Web  design  and,  most  likely,  were  not  familiar  with  this  term. 
Several  other  items  were  rewritten  in  order  to  make  the  questions  more  specific  and  valid. 

After  the  revisions  to  the  questionnaire  were  completed,  the  instrument  was  administered  to  a sample  of  53 
graduate  students.  The  reliability  was  analyzed  using  the  Cronbach  alpha  coefficient.  The  instrument’s  total 
reliability  was  high  (.95).  The  subscale  reliability  for  access  was  .65,  for  communication  .65,  for  content  .90,  for 
design  .85,  for  support  .75,  for  technology  .66,  for  outcomes  .76,  and  for  general  .79. 

The  second  sample,  consisting  of  teachers  and  administrators,  completed  the  questionnaire  during  the 
follow-up  study.  Once  again,  the  instrument’s  overall  reliability  was  high  (.97).  The  reliability  of  the  eight  subscales 
was  as  follows:  (a)  access,  .94,  (b)  communication,  .65,  (c)  content,  .92,  (d)  design,  .94,  (e)  support,  .54,  (f) 
technology,  .89,  (g)  outcomes,  .79,  and  (h)  general,  .90. 

Discussion 

The  internal  reliability  coefficient  for  the  subscale  support  on  the  WBTEQ  was  low  (.54)  in  the  follow-up 
study,  whereas  it  was  .75  in  the  pilot  study,  which  may  be  because  the  subscale  only  had  three  questions,  two  of 
which  were  marked  not  applicable  by  62.5%  and  69.6%  of  respondents  in  the  follo\wup  study.  Only  nine  and  seven 
respondents,  respectively,  answered  these  questions.  In  order  to  have  a valid  reliability  analysis  completed,  a 
researcher  needs  at  least  20  cases.  The  reader  is  therefore  advised  to  disregard  the  low  subscale  reliability  coefficient 
in  the  follow-up  study. 

It  is  possible  Item  42  on  the  WBTEQ,  “I  was  anxious  to  change  my  methods  when  I completed  the 
module,”  might  have  been  an  invalid  item.  During  the  first  instrument  administration,  one  graduate  student 
commented  on  how  the  word  anxious  had  a negative  connotation.  The  researcher  did  not  change  the  item  because  it 
did  not  cause  any  difficulties  during  the  pilot  study;  however,  it  may  be  the  reason  participants  in  the  follow-up 
study  disagreed  with  the  item.  Forty-five  percent  indicated  they  were  not  anxious  to  change  their  methods  after  the 
completion  of  the  module  (M  = 2.70).  The  question  was  revised  to  read  excited  after  the  study  was  completed. 
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Conclusion 

The  results  of  the  reliability  of  the  WBTEQ  are  promising  and  show  the  internal  consistency  reliability  for 
the  instrument  and  its  subscales  are  acceptable.  The  researcher  hopes  others  will  use  this  instrument  in  the  future  to 
evaluate  online  modules  and  courses  to  validate  the  findings  of  this  study.  Because  the  evaluation  of  online  courses 
is  an  important  issue,  we  should  continue  evaluating  our  courses  on  a continuous  basis.  At  the  very  least,  courses 
should  be  evaluated  after  participants  complete  the  course.  If  possible,  the  completion  of  a formative  evaluation  is 
highly  desirable. 

References 

Aggarwal,  A.  (2000).  Web-based  learning  and  teaching  technologies:  Opportunities  and  challenges. 
Hershey,  PA:  Idea  Group  Publishing. 

American  Society  for  Training  and  Development.  (2001,  June).  A vision  of  e -learning  for  America’s 
workforce. 

Alexandria,  VA:  Authors.  Retrieved  July  23,  2001,  from  http://www.astd.org 

Bastiaens,  T.  J.,  & Martens,  R.  L.  (2000).  Conditions  for  Web-based  learning  with  real  events.  In  B.  Abbey 
(Ed.),  Instructional  and  cognitive  impacts  of  Web-based  education  (pp.  1-31).  Hershey,  PA:  Idea  Group  Publishing. 
Belanger,  E.,  & Jordan,  D.  H.  (2000).  Evaluation  and  implementation  of  distance  learning:  Technologies,  tools  and 
techniques.  Hershey,  PA:  Idea  Publishing  Group. 

Berry,  L.  H.  (2000).  Cognitive  effects  of  Web  page  design.  In  B.  Abbey  (Ed.),  Instructional  and  cognitive 
impacts  of  Web-based  education  (pp.  41-55).  Hershey,  PA:  Idea  Publishing  Group. 

Biner,  P.  M.  (1993).  The  development  of  an  instrument  to  measure  student  attitudes  toward  televised 
courses.  The  American  Journal  of  Distance  Education,  4 (1),  62-73. 

Cohen,  A.  (2001,  January  2).  Corporate  e-ducation.  PC  Magazine,  137.  Retrieved  June  14,  2001,  from 
WebLUIS  online  database  (Academic  Index,  Item  RI67546519) 

Crocker,  L.,  & Algina,  J.  (1986).  Introduction  to  classical  and  modern  test  theory.  New  York:  Holt, 
Rinehart  and  Winston. 

Cronbach,  L.  J.  (1946).  Response  sets  and  test  validity.  Educational  and  PsychologicalMeasurement,  6, 

475-494. 

Dean,  P.  J.,  & Ripley,  D.  E.  (Eds.).  (1998).  Performance  improvement  interventions.  Washington,  DC: 
International  Society  for  Performance  Improvement. 

Gall,  M.  D.,  Borg,  W.  R.,  & Gall,  J.  P.  (1996).  Educational  research:  An  introduction  (6th  ed.).  White 
Plains,  NY:  Longman  Publishers. 

Harrison,  N.  (1999).  How  to  design  self-directed  and  distance  learning  programs.  Boston,  MA:  Me  Craw- 

Hill. 

Kemp,  J.  E.,  Morrison,  G.  R.,  & Ross,  S.  M.  (1998).  Designing  effective  instruction  (2nd  ed.).  Upper 
Saddle  River,  NJ:  Prentice-Hall. 

Kerlinger,  E.  N.,  & Lee,  H.  B.  (2000).  Foundation  of  behavioral  research  (4th  ed.).  Orlando,  EL:  Harcourt 
College  Publishers. 

Khan,  B.  H.  (1997).  Web-based  instruction.  Englewood  Cliffs,  NJ:  Educational  Technology  Publications. 
Kirkpatrick,  D.  L.  (1996).  Evaluating  training  programs:  The  four  levels.  San  Erancisco,  CA:  Berrett- 
Koehler  Publishers. 

Knowles,  M.  S.  (1989).  The  making  of  an  adult  educator.  San  Erancisco,  CA:  Jossey-Bass. 

Lau,  L.  (Ed.).  (2000).  Distance  learning  technologies:  Issues,  trends,  and  opportunities.  Hershey,  PA:  Idea 
Group  Publishing. 

Mantyla,  K.,  & Gividen,  J.  R.  (1997).  Distance  learning:  A step-by-step  guide  for  trainers.  Alexandria, 

VA:  American  Society  for  Training  and  Development. 

Mercer,  C.,  & Durham,  T.  W.  (2001).  The  Hood  Mysticism  Scale:  Does  presence  of  a neutral  response  item 
affect  response  style?  Psychological  Reports,  88,  335-338. 

Pearson,  L.  C.,  & Carey,  L.  M.  (1995).  The  academic  motivation  profile  for  undergraduate  student  use  in 
evaluating  college  courses.  Journal  of  Educational  Research,  88,  220-227. 

Schlough,  S.,  & Bhuripanyo,  S.  (1998).  The  development  and  evaluation  of  the  Internet  delivery  of  the 
Course  “Task  Analysis.  ” Paper  presented  at  the  meeting  of  the  Society  for  Information  Technology  & Teacher 
Education  International  Conference,  Washington,  DC.  (ERIC  Document  Reproduction  Service  No.  ED421089) 


47 


White,  S.  E.  (1999).  The  effectiveness  of  Web-based  instruction:  A case  study.  Paper  presented  at  the  joint 
meeting  of  the  Central  States  Communication  Association  and  the  Southern  States  Communication  Association,  St. 
Louis,  MO.  (ERIC  Document  Reproduction  Service  No.  ED430261) 

Wideman,  H.,  & Owston,  R.  D.  (1999).  Internet-based  courses  at  Atkinson  College:  An  initial  assessment 
(Tech.  Rep.  No.  1).  Toronto,  Canada:  York  University,  Centre  for  the  Study  of  Computers  in  Education.  Retrieved 
August  14,  2001,  from  http://www.edu.yorku.ca/csce/tech99-l.html 


48 


The  Affects  of  an  Online  Course  on  Turkish  Undergraduate  Students’ 
Perceived  Computer  Self-Efficacy  and  General  Self-Efficacy 


Mujgan  Bozkaya 
Anadolu  University 
Eskisehir,  Turkey 


Abstract 

This  paper  is  primarily  concerned  with  a study  on  online  student’s  self-efficacy.  The  study  focuses  on 
differences  between  Turkish  undergraduate  students'  general  self-efficacy  levels  and  computer  self-efficacy  levels  as 
well  as  determining  the  effects  of  an  online  course  on  the  students ' perceived  computer  self-efficacy  and  general 
self-efficacy  levels.  This  online  course  was  their  first  experience  in  online  learning.  The  participants  were  asked  to 
fill  out  a generalized  self-efficacy  scale  and  a computer  user  self-efficacy  scale  prior  and  posterior  to  the  course. 

The  pretest  results  have  revealed  that  Turkish  students’  generalized  self-efficacy  is  higher  than  their  computer  self- 
efficacy  scores  and  there  is  a significant  difference  between  these  scores.  The  posttest  results  have  also  shown  the 
same  difference.  However  no  significant  effect  of  online  course  has  been  identified. 


Introduction 

Bandura  (1986,  p.391)  defines  self-efficacy  as  belief  about  one’s  own  capabilities  to  organize  and  execute  a 
certain  task.  In  other  words,  self-efficacy  is  about  beliefs  that  are  related  to  the  consequences  of  individuals’  actions. 
An  individual’s  self-efficacy  highly  depends  on  prior  experiences  (success  or  failure),  others’  experiences 
(observing  others’  success  or  failure),  verbal  persuasion  (of  spouse,  friend,  etc)  or  emotional  states  (anxiety,  fear, 
etc)  (Cassidy  & Eachus,  2001,  p.2). 

Cassidy  and  Eachus  (2001)  indicate  that  self-  efficacy  levels  have  been  shown  to  be  related  to  choice  of 
task,  motivational  level  and  effort  and  perseverance  with  the  task.  Because  self-efficacy  is  based  on  self  perceptions 
regarding  particular  behaviors,  the  construct  is  considered  to  be  situation  specific  or  domain  sensitive.  In  other 
words,  a person  may  exhibit  high  levels  of  self-efficacy  within  one  domain  while  exhibiting  low  levels  of  self- 
efficacy  within  another  domain  (p.2). 

In  the  literature,  there  are  many  studies  (e.g.  Joo,  Bong  & Choi,  2000;  Pajeres,  1996)  about  measuring 
individual’s  levels  of  self  efficacy  within  different  domains  including  education  and  their  effects  on  different 
variables  such  as  achievement,  satisfaction,  motivation,  etc. 

However,  few  studies  that  explore  the  relationships  between  individual’s  levels  of  self-efficacies  specifics 
to  different  domains  have  been  conducted.  Eor  example,  no  study  has  been  found  in  the  literature  that  looks  for  the 
relationship  between  students’  level  of  general  self-efficacy  and  their  level  of  computer  self-efficacy. 

The  study  summarized  in  this  paper  investigates  this  relationship.  That  is,  it  looks  at  the  relationship 
between  Turkish  undergraduate  students’  general  self-efficacy  levels  and  their  computer  self-efficacy  levels.  It  also 
examines  the  effects  of  an  online  course  on  the  students’  general  self-efficacy  and  computer  self-efficacy.  Therefore 
the  research  questions  of  the  study  were  formulated  as: 

1.  Is  there  a difference  between  Turkish  undergraduate  students’  levels  of  self-efficacy  and  their  levels  of 
computer  self-efficacy? 

2.  What  are  the  effects  of  an  online  course  on  Turkish  undergraduate  students’  levels  of  self-efficacy  and 
computer  self-efficacy? 

3.  Is  there  a difference  in  effects  of  online  course  on  levels  of  self-efficacy  and  levels  of  computer  self- 
efficacy  of  the  Turkish  undergraduate  female  students  and  male  students? 


Method 

In  order  to  determine  the  levels  of  the  students’  computer  self-efficacy  and  general  self-efficacy,  two 
different  scales  were  combined  into  one  instrument.  “The  Computer  User  Self-efficacy  Scale’’  developed  by  Cassidy 
and  Eachus  (2001)  was  used  to  measure  the  students’  computer  self-efficacy  levels.  The  first  part  of  this  scale  that 
consisted  of  questions  related  to  demographic  information  about  the  participants  was  not  used.  The  second  part  that 
is  a 30-item  Likert  scale  was  used  as  one  the  instruments  of  the  study.  Eor  the  general  self-efficacy  levels  of  the 
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students,  another  scale,  “General  Perceived  Self-Efficacy  Scale’’  developed  by  Jerusalem  and  Schwarzer  in  1981  has 
been  chosen.  The  Generalized  Self-Efficacy  Scale  is  a 10-item  psychometric  scale  that  is  designed  to  assess 
optimistic  self-beliefs  to  cope  with  a variety  of  difficult  demands  in  life  (Schwarzer,  1992). 

The  subjects  of  this  study  were  the  students  who  voluntarily  take  the  “EIP206  Introduction  to  Educational 
Media’’  course  online  and  participate  the  study  in  Anadolu  University  School  of  Communication  Sciences 
Educational  Communications  and  Planning  Department  in  Spring  2002.  This  course  was  their  first  experience  in 
online  learning.  Although  35  students  accepted  and  participated  the  study  at  the  beginning,  only  28  of  these  students 
were  completed  the  study. 

The  pretest  was  administered  at  the  beginning  of  the  semester  and  the  data  derived  from  this  test  were 
analyzed  for  the  first  question  of  the  study.  Same  instrument  was  used  as  posttest  at  the  end  of  the  semester  and  the 
results  were  compared  with  the  pretest  results  for  the  last  two  questions  of  the  study. 

The  reliability  of  computer  self-efficacy  scale  was  found  as  0.79  and  general  perceived  self-efficacy  scale 
was  as  0.82. 


Results 

A t-test  analysis  was  used  to  see  the  difference  between  general  and  computer  self-efficacy  scores  of 
Turkish  undergraduate  students.  Table  1 shows  the  mean  scores  of  subjects’  levels  of  general  self-efficacy  and 
computer  self-efficacy  in  pretest  and  the  results  of  the  t-test  analysis.  In  addition  Table  1 includes  mean  scores  and  t- 
test  analysis  in  posttest.  The  data  have  revealed  that  students’  perceived  general  self-efficacy  mean  (M=4.69)  is 
higher  then  their  computer  self  efficacy  mean  (3.87)  and  there  is  a statically  significant  difference  among  these 
scores  (p<.0001).  Same  results  can  be  seen  in  posttest,  too.  In  other  words,  students’  perceived  general  self-efficacy 
mean  (M=4.77)  is  also  higher  than  their  computer  self-efficacy  mean  (M=3.63)  and  there  is  a significant  difference 
among  these  two  scores  (p<0.0001)  after  the  online  course. 


Table  1:  Mean  scores  of  all  students  in  pretest  and  posttest 


Tests 

Mean 

General 

Self-Efficacy 

Computer 

Self-Efficacy 

N 

t 

Pretest 

4.69 

3.87 

28 

.000** 

Posttest 

A.ll 

3.63 

28 

.000** 

*p<.05  **p<.01 

In  order  to  see  whether  there  is  relation  between  general  self-efficacy  and  computer  self-efficacy  scores  of  students 
in  both  tests,  a Pearson  correlation  analysis  can  be  used.  Table  2 shows  the  results  of  this  analysis.  According  to  this 
analysis  there  is  a significant  strong  relation  (r=0.703,  N=28,  p<0.000)  only  between  pretest  general  self-efficacy 
scores  and  posttest  general  self-efficacy  scores. 


Table  2:  Correlation  coefficients  between  in  both  tests  and  general  self-efficacy  and  computer  self-efficacy 


Pretest 

Computer  Self- 
Efficacy 

Pretest 
General  Self- 
Efficacy 

Posttest 
Computer  Self- 
Efficacy 

Posttest 
General  Self- 
Efficacy 

Pretest  Pearson  Correlation 

1,000 

,256 

,152 

,364 

Computer  Self-Efficacy  Sig. (2 -tailed) 

,189 

,441 

,057 

N 

28 

28 

28 

28 
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Pretest 

Pearson  Correlation 

,256 

1,000 

,184 

,703** 

General  Self-Efficacy 

Sig.  (2-tailed) 

,189 

,350 

,000 

N 

28 

28 

28 

28 

Posttest 

Pearson  Correlation 

,152 

,184 

1,000 

,142 

Computer  Self-Efficacy 

Sig.  (2-tailed) 

,441 

,350 

,472 

N 

28 

28 

28 

28 

Posttest 

Pearson  Correlation 

,364 

,703** 

,142 

1,000 

General  Self-Efficacy 

Sig.  (2-tailed) 

,057 

,000 

,472 

N 

28 

28 

28 

28 

**  Correlation  is  significant  at  the  0.01  level  (2 -tailed) 


On  the  other  hand,  another  t-test  analysis  is  used  to  investigate  the  effects  of  online  course  on  students’  general  self- 
efficacy  and  computer  self-efficacy  levels.  The  t-test  results  indicated  no  significant  in  general  self-efficacy  and 
computer  self-efficacy  scores  between  pretest  and  post  test.  Table  3 presents  the  means  scores  and  the  results  of  the 
t-test  analysis. 

Table  3:  Mean  scores  of  all  students  in  both  tests  and  results  of  the  t-test  analysis 


Self  Efficacy 

Tests 

Mean 

SD 

N 

t 

General  Self-Efficacy 

Pretest 

4,69 

.657 

28 

.478 

Posttest 

4,77 

.843 

Computer  Self-Efficacy 

Pretest 

3,87 

.510 

28 

.060 

Posttest 

3,63 

.511 

*p<  .05 


In  addition.  Table  4 illustrates  the  mean  scores  of  female  and  male  students  in  both  tests  and  the 
comparison  of  these  scores.  The  comparison  shows  that  the  mean  scores  of  female  students  were  always  lower  than 
that  of  male  student  in  pretest  and  posttest.  Additionally  there  was  no  significant  difference  between  pretest  and 
posttest  mean  scores  either  general  self-efficacy  or  computer  self-efficacy  of  female  students. 

However,  the  mean  score  of  male  students  in  general  self-efficacy  scale  was  showed  significant  change 
between  pretest  and  posttest  (p<  .007). 


Table  4:  Mean  scores  of  female  and  male  students  in  both  tests  and  results  of  the  t-test  analysis 


Tests 

Sex 

Mean 

Pretest  Posttest 

N 

t 

Eemale 

4,48 

4,34 

17 

.379 

General  Self-Efficacy 

Male 

5,01 

5,43 

11 

.007* 

Eemale 

3,71 

3,51 

17 

.221 

Computer  Self-Efficacy 

Male 

4,12 

3,8 

11 

.172 

*p<  .05  **p<.01 
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Discussion 

According  to  results  of  this  study,  it  can  be  drawn  that  there  is  significant  difference  between  students’ 
general  self-efficacy  and  computer  self-efficacy.  These  results  support  the  literature.  In  other  words,  a person  may 
exhibit  high  levels  of  general  self-efficacy  while  exhibiting  low  levels  of  computer  self-efficacy. 

Also,  the  study  demonstrated  that  the  online  course  caused  a few  increase  in  students  levels  of  general  self 
efficacy  but  it  is  not  statistically  significant.  Interestingly  the  students’  levels  of  computer  self-efficacy  decreased  at 
the  end  of  course.  However,  it  is  not  significant,  too.  The  instructor  of  the  course  indicated  that  a few  technical 
problems  happened  during  the  implementation  of  the  course  and  the  students  ejqtressed  their  discomfort  about  them. 
So  this  might  be  a cause  of  this  decrease.  In  addition,  this  course  was  their  first  course.  So  they  might  not  see  the 
potential  of  the  online  learning  in  this  course. 

This  decrease  in  computer  self-efficacy  levels  of  the  students  is  not  sex  related  and  not  statistically 
significant.  But,  the  increase  in  general  self-efficacy  is  sex  related.  In  other  words  male  students’  general  self- 
efficacy  scores  have  shown  a significant  increase  while  female  students’  general  self  efficacy  scores  have  shown  a 
minor  decrease  which  is  not  statically  significant.  This  result  might  be  interpreted  as  that  male  student  hesitate 
before  an  experience  and  show  greater  self-efficacy  after  an  experience. 

More  research  with  a large  number  of  subjects  should  be  conducted  to  investigate  the  effects  of  online 
learning  on  students’  self-efficacy  levels.  Also,  relations  between  computer  experience,  computer  ownership  and 
effects  of  online  education  on  self-efficacy  should  be  examined. 
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Abstract 

Many  school  districts  (SDs)  use  the  World  Wide  Web  (WWW)  to  disseminate  a wide  variety  of  information 
about  things  such  as  district  events,  policies,  and  a wide  variety  of  student  information.  On-line  barriers  limit  the 
accessibility  of  the  WWW  for  persons  and  students  with  disabilities  and  thus  can  limit  their  access  to  vital 
information.  The  purpose  of  this  study  was  to  evaluate  the  accessibility  ofSD  home  pages  in  the  United  States  and 
Canada.  A total  of  120  SD  Web  sites  were  located  using  a popular  online  school  directory  and  evaluated  for 
accessibility.  A software  program  was  used  to  quantify  the  number  of  accessibility  errors  at  each  site.  The  results 
indicated  that  most  (74.3%)  SD  home  pages  had  accessibility  problems,  and  the  majority  of  these  problems  were 
severe  problems  that  should  be  given  a high  priority  for  correcting.  The  good  news  is  that  the  majority  of  the  errors 
can  easily  be  corrected.  The  work  reflects  a need  for  SDs  to  examine  the  accessibility  of  their  home  pages. 
Recommendations  for  improving  accessibility  is  provided. 


School  districts’  web  sites:  How  accessible  are  they? 

Since  the  mid  1990’s  the  World  Wide  Web  (WWW)  has  become  an  important  way  to  disseminate 
information  about  a wide  variety  of  organizations.  These  organizations  include  many  educational  institutions  such 
as  Universities,  departments  of  public  instruction,  and  individual  schools.  All  of  these  institutions’  policies 
concerning  equal  access  are  guided  by  federal  laws  such  as  The  Americans  with  Disabilities  Act  (ADA).  The  ADA, 
enacted  in  1990,  provides  the  same  civil  rights  protection  to  individuals  with  disabilities  as  other  federal  laws  that 
prevent  discrimination  on  the  basis  of  race,  gender,  national  origin,  and  religion  (Button  & Wobschall,  1994).  Title 
III  of  the  ADA  directs  that  public  facilities  make  reasonable  modifications  to  control  discrimination  and  support 
accessibility  in  policies,  practices,  and  procedures  (Council  for  Exceptional  Children,  1994).  As  a result  of  this 
landmark  legislation,  accessibility  alterations  like  providing  ramps  to  elevated  areas  and  providing  accessible 
signage  through  height  adjustments  and  raised  lettering  have  become  commonplace  across  the  country. 

A projected  $5.67  billion  was  spent  on  technology  in  America’s  public  schools  during  the  1999-2000 
school  year.  While  63%  of  those  dollars  were  spent  on  hardware,  it  appears  that  public  schools  have  shifted  their 
budgets  toward  software  and  staff  development.  More  than  46%  of  schools  reported  that  the  majority  of  their 
teachers  are  intermediate  level  users  of  technology  (able  to  use  a wide  variety  of  computer  applications).  More  than 
60%  of  schools  reported  that  the  majority  of  teachers  use  the  WWW  for  instructional  purposes,  and  82%  of  schools 
provide  WWW  access  in  classrooms. 

Fifty-two  percent  of  U.S.  homes  with  children  ages  2-  17  have  WWW  connections  at  home,  up  form  just 
15%  just  two  years  ago.  In  addition,  20%  of  students  ages  8-16  have  computers  in  their  bedrooms,  with  more  than 
half  hooked  up  to  the  WWW.  As  the  percentage  of  WWW  usage  continues  to  increase  both  at  school  and  at  home 
so  does  the  need  for  clearer  behavioral  guidelines  for  students.  The  following  is  a list  of  practical  steps  for  school 
administrators  that  address  student  usage  of  the  WWW  on  school  premises  and  at  home. 

1.  When  school  administrators  review  a situation,  he/she  should  first  determine  whether  the  fact 
pattern  actually  falls  within  the  code  of  student  behavior  set  at  each  particular  school.  Often,  the 
school  rules  may  fall  short  of  covering  new  situations.  Stretching  rules  to  fit  new  situations  is 
dangerous  and  risky  practice. 

2.  Before  school  administrators  moves  forward  with  discipline  for  off-site  internet  conduct,  he/she 
should  put  down  in  writing  the  connection  between  off-site  conduct  and  the  impact  that  it  has  or 
will  have  at  school.  School  administrators  should  always  investigate  to  see  if  the  student  violated 
any  discipline  code,  if  so,  this  might  provide  a stronger  basis  for  disciplinary  action. 

3.  Compare  WWW  behavior  with  its  non-WWW  behavioral  counterpart.  For  example,  if  the  issue  is 
vulgarity,  can  a student  be  held  accountable  under  the  behavioral  policy  set  by  the  school? 

4.  School  administrators  should  make  it  clear  to  all  students  that  your  staff  monitors  the  WWW  on  a 
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regular  basis. 

5.  School  administrators  should  enlist  the  aid  and  support  of  parents.  Parents  need  to  be  aware  of  the 
schools  WWW  policy.  Having  parents  sign  a “student  WWW  code  of  conduct’’  at  the  beginning  of 
each  school  year  will  help  parents  to  better  understand  what  the  school  deems  as  appropriate  and 
inappropriate. 

6.  School  administrators  should  have  a clear  set  of  procedures  that  staff  should  follow  if  they  believe 
a student  has  violated  the  WWW  code  of  conduct.  Staff  should  acquire  as  much  evidence  as 
possible. 

7.  School  administrators  may  consider  lodging  a complaint  to  the  WWW/internet  service  provider. 
Often  the  WWW/internet  service  provider  will  remove  offensive  material  especially  when  the 
material  is  sexually  explicit,  personally  degrading  to  target  specific  individuals,  or  racially  or 
sexually  harassing. 

Physical  barriers  are  obvious  accessibility  concerns  confronting  students  with  disabilities.  Web  site 
developers  need  to  be  just  as  aware  that  on-line  barriers  can  create  significant  accessibility  problems  for  some  users. 
The  Americans  with  Disabilities  Act  requires  that  all  organizations  make  reasonable  accommodations  for 
individuals  with  disabilities.  Section  508  of  the  Rehabilitation  Act  requires  that  all  organizations  receiving  US 
Federal  funds  must  comply  with  standards  that  make  electronic  equipment  and  Web  sites  usable  by  people  with 
sight,  hearing,  and  other  disabilities.  Using  the  WWW  as  a resource  for  distributing  information  is  no  exception. 

A variety  of  disabilities  can  reduce  accessibility  to  the  WWW.  Visual,  hearing,  movement,  cognitive, 
speech,  and  other  impairments  can  limit  availability  of  information.  Assisted  technologies  or  accessibility  aids  such 
as  Braille  output  systems,  modification  of  keyboards,  screen  enlargement  utilities,  voice  output  utilities,  and  other 
technologies  allow  students  with  disabilities  to  access  information  on  the  WWW.  However,  because  of  the 
complexity  of  many  Internet  resources,  some  information  cannot  be  accessed  with  these  aids.  Developers  of 
accessibility  aids  continue  to  identify  and  develop  features  that  can  overcome  some  of  these  barriers,  but  there  are 
many  things  that  Web  site  developers  can  do,  with  very  little  effort,  that  would  make  their  pages  more  accessible. 

The  Trace  Research  and  Development  Center  at  the  University  of  Wisconsin  at  Madison  produced  the 
Unified  Web  Site  Accessibility  Guidelines  (1999).  These  guidelines  were  transferred  to  the  Web  Accessibility 
Initiative  (WAI)  of  the  World  Wide  Web  Consortium  (W3C)  and  used  to  produce  the  Web  Content  Accessibility 
Guidelines  1.0  (Chisholm  & Vanderheiden,  1999a).  The  primary  goal  of  the  guidelines  is  to  promote  content 
accessibility.  The  guidelines  do  not  discourage  content  developers  from  using  images,  video,  and  other  multimedia 
tools,  but  rather  explain  how  to  make  multimedia  content  more  accessible  to  a wider  audience. 

According  to  the  guidelines,  measures  for  improving  accessibility  falls  into  the  following  categories:  (a) 
structure — HTML  documents  should  use  markup  to  convey  meaning  and  less  for  format  and  layout  pages;  (b) 
navigation — authors  should  support  keyboard -only  navigation  and  methods  to  facilitate  orientation;  and  (c) 
alternative  content — authors  should  always  provide  alternative  ways  to  access  information  presented  with  images, 
sounds,  applets,  and  scripts.  These  recommendations  have  been  categorized  as  Priority  1,  2,  and  3 errors.  Priority  1 
errors  involve  issues  that  make  it  impossible  for  one  or  more  groups  to  access  information  about  the  Web  site. 

These  issues  must  be  addressed  to  consider  the  Web  site  minimally  accessible.  Priority  2 errors  make  it  difficult  for 
users  to  access  Web  site  content.  Priority  3 errors  may  be  addressed  by  web  developers  and  make  it  somewhat 
difficult  for  readers  to  access  information  in  the  Webpage.  Additionally,  the  WAI  provides  specific 
recommendations  and  strategies  on  how  to  produce  Web  sites  that  are  in  agreement  with  the  guidelines.  Examples 
and  models  of  the  appropriate  use  of  HTML  tags  (i.e.,  page  title,  text,  lists  and  outlining,  tables,  links,  objects, 
images,  audio,  applet,  frames,  forms,  and  scripts)  and  elements  are  provided. 

The  Web  Content  Accessibility  Guidelines  1.0  (Chisholm  & Vanderheiden,  1999a)  document  is  organized 
around  two  general  themes  and  14  guidelines  or  general  principles  of  accessible  design  (see  Table  1).  The  themes 
are  (a)  ensuring  graceful  transformation  and  (b)  making  content  understandable  and  navigable.  The  document 
provides  the  rationale  behind  the  guidelines  and  describes  some  of  the  users  who  benefit  when  they  are  applied  to 
Web  sites.  In  addition,  a list  of  checkpoints  is  provided  that  explains  how  the  guidelines  apply  to  typical  content 
development  scenarios.  Each  checkpoint  is  specific  enough  to  be  verified  while  general  enough  to  allow  Web 
developers  freedom  to  use  appropriate  strategies. 
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Table  1.  Web  Content  Accessibility  Themes  and  Guidelines 


Theme 

Item 

Guideline 

Ensuring  Graceful  Transformation 

I 

Provide  equivalent  alternatives  to  auditory  and  visual  content. 

2 

Don’t  rely  on  color  alone 

3 

Proper  use  of  markup  and  style  sheets. 

4 

Clarify  natural  language  usage 

5 

Create  tables  that  transform  gracefully 

6 

Ensure  that  pages  featuring  new  technologies  transform  gracefully 

7 

Ensure  user  control  of  time  -sensitive  content  changes 

8 

Ensure  direct  accessibility  of  embedded  user  interfaces. 

9 

Design  for  device-independence. 

10 

Use  interim  solutions. 

Making  Content  Understandable  and 

II 

Use  W3C  technologies  and  guidelines 

Navigable 

12 

Provide  context  and  orientation  information 

13 

Provide  clear  navigation  mechanisms. 

14 

Ensure  that  documents  are  clear  and  simple 

The  guidelines  that  primarily  address  the  theme  of  ensuring  graceful  transformation,  Guidelines  1 through 
10,  assist  Web  developers  in  producing  sites  that  remain  accessible  despite  constraints  confronted  by  people  with 
disabilities.  For  example.  Guideline  1 states  that  Web  developers  should  provide  equivalent  alternatives  to  auditory 
and  visual  content.  Text  can  be  rendered  in  ways  that  are  available  to  almost  all  browsing  devices  and  accessible  to 
all  users,  but  auditory  and  visual  content  are  not.  Guidelines  1 1 through  14  primarily  address  the  theme  of  making 
content  understandable  and  navigable.  This  includes  providing  navigation  tools  and  orientation  information  in  pages 
with  maximize  accessibility  and  usability.  Not  all  users  can  make  use  of  visual  clues  such  as  image  maps  or 
graphical  information,  but  with  orientation  information,  users  can  understand  many  of  these  graphical  images.  Of 
course  the  information  presented  here  is  an  overview  of  these  guidelines  and  the  Web  Content  Accessibility 
Guidelines  1.0  (Chisholm  & Vanderheiden,  1999a)  document  should  be  consulted  for  more  detail  in  developing 
content -accessible  Web  sites. 

Building  Web  sites  that  comply  with  standards  for  accessibility  should  be  a high  priority  for  Web  site 
developers.  To  date,  little  research  has  documented  the  extent  to  which  accessibility  goals  have  been  reached. 


Method 

This  study  examined  the  accessibility  of  School  District  home  pages.  The  purpose  of  the  research  described 
in  this  article  was  to:  (a)  evaluate  the  accessibility  of  School  District  home  pages,  and  (b)  direct  readers  to  resources 
that  are  available  to  assist  in  the  development  of  accessible  home  pages. 

Sampling 

A list  of  URLs  for  School  District  Home  Pages  was  generated  using  the  Web  66  (2001)  website.  This 
online  resource  provides  a detailed  list  of  school  district  websites  in  the  United  States  and  Canada  as  well  as  around 
the  world.  A total  of  567  school  district  Web  sites  in  the  United  States  and  Canada  were  randomly  selected  from 
this  directory. 

Evaluation  Process 

Each  School  District’s  home  page  was  evaluated  using  Bobby  3.2  (Center  for  Applied  Special  Technology, 
2001),  a software  package  that  analyzes  Web  sites  in  accordance  with  the  W3C  Web  Accessibility  Initiative 
guidelines.  Results  from  Bobby  3.2  provide  a measure  of  the  extent  to  which  a Web  site  is  accessible  for  people 
with  disabilities.  The  type  of  accessibility  error  (e.g.,  images  without  alternative  text,  links  without  alternative  text, 
and  pages  not  usable  without  frame),  the  severity  of  the  error  (e.g..  Priority  1,  Priority  2,  Priority  3),  and  the  ease 
with  which  the  error  can  be  fixed  (e.g.,  easy,  moderate,  hard)  are  provided  in  a summary  report.  In  this  study  only 
the  initial  School  District  home  page  was  evaluated  and  no  links  within  the  domain  were  evaluated.  Scores  for  each 
home  page  were  tabulated  and  analyzed. 

There  are  many  accessibility  issues  that  Bobby  3.2  cannot  detect.  For  example,  Bobby  cannot  determine 
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programmatically  if  the  Web  site  is  following  accessibility  principles,  and  can  only  draw  the  users  attention  to  the 
potential  risks  of  any  technology  that  is  used.  The  potential  errors  will  be  reported  in  the  results,  but  the  researchers 
did  not  physically  ejamine  the  Web  site  to  evaluate  these  potential  errors. 


Results 

Of  the  567  web  pages  evaluated,  74.3%  of  the  Web  site  home  pages  had  at  least  one  accessibility  error.  The  means 
and  standard  deviations  for  the  accessibility  and  potential  accessibility  errors  sorted  by  priority  are  presented  in  Table  2. 

There  was  an  average  of  .91  Priority  1 accessibility  errors  on  the  School  District  home  pages.  This  indicates  significant 
accessibility  issues  that  can  hinder  the  reader’s  access  to  information  on  the  Webpage.  Priority  2 and  Priority  3 errors 
averaged  2.33  and  1.64.  While  these  errors  are  not  as  severe  as  Priority  1 errors,  they  can  still  affect  the  degree  to  which  a 
reader  can  access  a Web  site.  As  seen  in  Table  3 the  most  common  Priority  1 accessibility  problems  identified  in  home  pages 
of  School  Districts  were  (a)  using  alternate  ways  to  convey  information  represented  by  color  (74%),  (b)  providing  extended 
descriptions  of  alternate  text  (71%),  and  (c)  using  structural  markup  to  identify  their  hierarchy  and  relationship  of  two  or 
more  header  rows  or  columns  in  a table  (66%).  Using  alternate  ways  to  convey  information  represented  by  color  means  that 
information  represented  or  emphasized  by  specific  colors,  red  for  example,  should  be  conveyed  in  ways  that  can  be 
recognized  by  software  readers  that  convert  text  to  speech.  If  images  are  used  and  they  convey  information  then  extended 
descriptions  should  be  used  to  convey  this  information.  Finally,  the  relationship  of  multiple  rows  and  or  columns  used  for 
table  headers  should  be  described  so  that  software  readers  can  correctly  relay  the  relationship  of  the  information  among  these 
elements.  As  will  be  seen  later,  many  of  these  problems  were  rated  as  easy  to  fix.  Priority  2 errors  reported  included 
sufficient  contrast  between  foreground  and  background  colors  (81%),  lack  of  descriptive  titles  to  links  (78%),  and  use  of 
movement  in  images  (64%).  The  most  frequent  Priority  3 errors  reported  included  no  identification  the  language  of  the  text 
(96%),  lack  of  keyboard  shortcuts  to  frequently  used  links  (80%),  and  no  logical  tab  order  among  page  elements  (80%). 

Table  2.  Accessibility  Errors  Categorized  by  Priority 


Accessibility  Errors 


Severity 

N 

Minimum 

Maximum 

M 

SD 

Priority  1 

567 

0 

4 

.91 

.67 

Priority  2 

567 

0 

6 

2.33 

1.57 

Priority  3 

567 

0 

3 

1.64 

.57 

Table  3.  Type  Accessibility  Error,  Percentage  of  Homepages  with  Error,  and  Ease  of  Fixing  Error 

Type  of  Accessibility  Error  Percent  Ease  ToEix 

Priority  One  Errors 

If  you  use  color  to  convey  information,  make  sure  the  information 


is  also  represented  another  way. 

If  an  image  conveys  important  information  beyond  what  is  its 

74 

Moderate 

alternative  text,  provide  an  extended  description. 

If  a table  has  two  or  more  rows  or  columns  that  serve  as  headers. 

71 

Moderate 

use  structural  markup  to  identify  their  hierarchy  and  relationship. 

66 

Moderate 

Provide  alternative  text  for  all  images. 

Eor  tables  not  used  for  layout  (for  example,  a spreadsheet). 

62 

Easy 

identify  headers  for  the  table  rows  and  columns. 

53 

Easy 

Priority  Two  Errors 

Check  that  the  foreground  and  background  colors  contrast  sufficiently 
with  each  other. 

81 

Easy 

Add  a descriptive  title  to  links  when  needed. 

78 

Easy 

Avoid  use  of  deprecated  language  features  if  possible. 

Avoid  using  tables  to  format  text  documents  in  columns  unless 

74 

Moderate 

the  table  can  be  linearized. 

66 

Hard 

Avoid  using  movement  in  images  where  possible. 

Use  relative  sizing  and  positioning  (%  values)  rather  than  absolute 

64 

Easy 

(pixels).  63 

Moderate 
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Priority  Three  Errors 

Specify  a logical  tab  order  among  form  controls,  links  and  objects.  80  Moderate 

Consider  adding  keyboard  shortcuts  to  frequently  used  links.  80  Moderate 


Table  3 also  indicates  that  the  majority  of  accessibility  errors  were  rated  as  easy  to  fix.  Examples  of  these 
types  of  errors  include  alternate  text  for  images,  and  specifying  the  relationship  among  multiple  headers  in  tables. 
Some  suggested  ways  to  fix  these  errors  include  the  use  of  the  ALT  tag  and  descriptive  text  with  every  image  to 
present  textual  information  about  images.  A similar  technique  can  be  employed  to  provide  descriptive  information 
about  links. 

Discussion 

The  Americans  with  Disabilities  Act  directs  that  individuals  with  disabilities  including  students  and  parents 
being  served  by  School  Districts  are  entitled  to  the  same  civil  rights  protections  as  their  neighbors  and  peers  without 
disabilities.  The  law  has  come  to  be  associated  with  efforts  to  make  public  and  private  facilities  and  institutions 
more  accessible  for  individuals  with  disabilities.  The  Internet  and  World  Wide  Web  have  revolutionized  access  to 
the  resources  and  services  of  American  businesses,  public  and  private  institutions,  and  other  organizations.  The 
accessibility  of  this  information  to  students  with  disabilities  has  not  been  extensively  studied. 

School  Districts  use  the  WWW  to  disseminate  and  gather  information.  On-line  barriers  limit  the 
accessibility  of  the  WWW  for  individuals  with  disabilities.  The  purpose  of  this  study  was  to  evaluate  the 
accessibility  of  School  Districts’  home  pages.  567  Web  sites  were  randomly  selected  for  evaluation.  Bobby  3.2,  a 
software  program,  was  used  to  quantify  the  number  of  accessibility  errors  at  each  site.  Most  School  Districts’  home 
pages  had  accessibility  problems  (74.9%).  Most  of  the  errors  were  rated  as  severe  and  should  be  given  a high 
priority.  The  good  news  is  that  the  majority  of  the  errors  can  easily  be  corrected.  The  need  for  School  Districts  to 
examine  the  accessibility  of  their  home  pages  is  evident  in  the  outcomes. 

School  District  Web  developers  need  to  examine  their  Web  sites  for  accessibility  problems.  It  is  strongly 
recommended  that  validation  methods  be  used  in  the  early  stages  of  Web  development  that  will  help  make  problems 
easier  to  correct  and  assist  developers  in  avoiding  accessibility  problems.  There  are  two  suggested  methods  of 
validating  a Web  site  for  accessibility  (Chisholm  & Vanderheiden,  1999a).  Eirst,  automatic  tools  are  available  for 
scanning  the  site  and  providing  data.  Bobby  3.2  and  other  validation  services  should  be  used  to  provide  information 
concerning  accessibility  problems.  Automatic  tools  are  convenient  but  do  not  identify  all  accessibility  issues; 
therefore,  it  is  recommended  that  each  site  be  examined  by  a knowledgeable  individual  and  individuals  with 
disabilities  to  ensure  clarity  of  language  and  ease  of  navigation.  The  processes  of  rapid  prototyping  and  formative 
evaluation  have  been  used  for  many  years  to  help  develop  educational  software  and  have  proven  to  be  useful  in  the 
process  of  Web  site  development  (Corry,  Erick,  & Hansen,  1997).  Expert  and  novice  users  with  disabilities  should 
be  invited  to  view  home  pages  and  provide  feedback  about  the  seveity  of  accessibility  or  usability  problems. 

One  global  suggestion  for  the  Web  site  designer  is  that  all  Web  pages  should  be  encoded  for  meaning 
rather  than  appearance.  Eor  example,  providing  alternative  ways  of  obtaining  information  is  a key  to  overcoming 
many  accessibility  errors.  By  using  ALT=”TEXT”  tags  on  all  images,  approximately  one-third  of  the  errors 
discovered  in  this  study  would  be  fixed. 

There  are  many  Web  sites  that  provide  information  and  recommendations  for  accessible  Web  pages.  Below 
are  a few  recommendations  for  evaluating  Web  sites  for  accessibility: 

1.  Web  site  designers  should  follow  the  accessibility  guidelines.  There  are  several  sites  on  the  WWW  that  give 
recommendations . 

2.  Each  Web  page  should  be  tested  for  accessibility.  Several  different  procedures  should  be  used:  (a)  view  each 
page  on  monochrome  screen  or  use  high  contrast  option  of  control  panel;  (b)  turn  off  graphics  and  view  page 
for  readability  or  use  Lynx  to  view  the  page;  (c)  select  only  text,  print  to  clipboard,  and  view  for  readability;  (d) 
navigate  using  only  the  keyboard;  and  (e)  use  Bobby  or  other  Web  evaluation  software  to  test  the  Web  pages. 

3.  Web  site  designers  should  provide  a “text  only’’  version  of  the  CE’s  Web  site.  This  can  be  done  using 
cascading  style  sheets  so  that  updates  can  be  accomplished  more  easily.  Providing  a text  only  version  of  a Web 
site  also  provides  users  with  slow  internet  connections  with  a relatively  fast  method  of  accessing  an  institution’s 
Web  site. 

4.  Web  site  designers  should  provide  the  reader  with  alternative  methods  for  obtaining  information  about  the  CE 
Program.  Providing  a phone  number,  email,  or  mailing  address  can  do  this.  This  information  should  be 
displayed  in  a prominent  place  on  the  CE’s  Web  site. 
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Page  authors  should  not  produce  “handicapped  products”  to  make  Web  sites  accessible.  Every  effort  should  be 
made  to  keep  all  Web  sites  in  the  mainstream  and  provide  elements  that  allow  universal  access.  Accessibility 
guidelines  are  not  designed  to  stifle  the  creative  freedom  of  Web  site  designers;  however,  extra  thought  and  effort  is 
required  in  designing  accessible  Web  sites. 

Currently,  several  tools  and  standards  are  being  developed  to  help  Web  developers  in  these  efforts.  In 
Microsoft  Corporation  (2001)  has  recently  announced  its  plans  to  help  developers  using  FrontPage,  a popular  web 
development  tool,  to  make  their  websites  conform  to  the  latest  US  federal  guidelines.  The  World  Wide  Web 
consortium  has  also  announced  guidelines  for  software  tools  to  help  persons  with  disabilities  access  a variety  of 
multimedia  content  (World  Wide  Web  Consortium,  2001a).  Finally,  at  the  time  of  this  writing,  the  World  Wide 
Web  consortium  is  working  on  the  second  version  of  Web  Accessibility  Standards  (World  Wide  Web  Consortium, 
2001b).  These  new  standards  will  not  only  help  authors  create  accessible  Web  sites  but  will  also  improve 
accessibility  of  the  web  for  persons  with  disabilities. 


References 

American  Counseling  Association.  (June,  1995).  Code  of  ethics  and  standards  o/pracf/cg;  Alexandria,  VA: 

Author. 

Button,  C.,  & Wobschall,  R.  (1994).  The  Americans  with  Disabilities  Act  and  assistive  technology.  Journal 
of  Vocational  Rehabilitation,  4,  196-201. 

Capozzi,  D.  (1998).  Accessibility  Day  1998  Speech. 

Center  for  Applied  Special  Technology  (2001).  Bobby  3.2  [computer  program].  Available: 
http://www.cast.org/bobby/ 

Chalfen,  D.  H.,  & Farb,  S.  E.  (1996).  Universal  access  and  the  ADA:  A disability  access  design 
specification  for  the  new  UCLA  library  online  information  system.  Library  Hi  Tech,  14{\),  51-56. 

Chisholm,  W.,  & Vanderheiden,  G.  (1999a).  Web  Content  Accessibility  Guidelines  1.0  [on-line].  Available: 
http://www.w3.org/TR/WAI-WEBCONTENT/. 

Chisholm,  W.,  & Vanderheiden,  G.  (1999b).  Techniques  for  Web  Content  Accessibility  Guidelines  7.0  [on- 
line]. Available:  http://www.w3.org/TR/WCAG10-TECHS/ 

Corry,  M.D.,  Frick,  T.W.,  & Hansen,  L.  (1997).  User-centered  design  and  usability  testing  of  a web  site: 

An  illustrative  case  study.  Educational  Technology  Research  and  Development  45 (4),  65-72. 

Council  for  Accreditation  of  Counseling  and  Related  Educational  Programs.  (2001).  The  2001  Standards. 
http://www.counseling.org/cacrep/2001standards700.htm. 

Council  for  Exceptional  Children.  (1994).  Planning  accessible  conferences  and  meeting:  An  ERIC/OSEP 
information  brief  for  conference  planners.  Reston,  VA:  Author. 

Hall,  G.  C.,  & Barongan,  C.  (2002).  Multicultural  counseling.  Upper  Saddle  River,  JN:  Prentice  Hall. 

Henwood,  P.G.,  & Pope-Davis,  D.  B.  (1994).  Disabilty  as  cultural  diversity:  Counseling  the  hearing 
impaired.  Counseling  Psychologist,  22, 489-503. 

Mendle,  J.  (1995).  Library  services  for  persons  with  disabilities.  Reference  Librarian,  49-50,  105-121. 

Microsoft  Corporation  (2001).  Microsoft  Announces  Strategic  Relationship  To  Offer  HiSoftware's  New 
Web  Site  Accessibility  Verification  Tool  For  FrontPage  [On-line].  Available: 
http://www.microsoft.eom/PressPass/press/2001/Sep01/09-24HiSoftwarePR.asp. 

Middleton,  R.A.,  Rollins,  C.  W.,  & Harley,  D.A.  (1999).  The  historical  and  political  context  of  the  civil 
rights  of  persons  with  disabilities:  A multicultural  perspective  for  counselors.  Journal  of  Multicultural  Counseling 
and  Development,  27,  105-120. 

Osfield,  K.  (1993).  Americans  with  Disabilities  Education  Act  (ADA):  Self-evaluation  for  the  University  of 
Florida.  Gainesville,  EL:  University  of  Florida. 

Swartz- Kulstad,  J.L.,  & Martin,  W.E.  (1999).  Impact  of  culture  and  context  on  psychosocial  adaptation: 

The  cultural  and  contextual  guide  process.  Journal  of  Counseling  and  Development,  77,  281-293. 

Vanderheiden,  G.C.  & Chisholm,  W.A.  (1998).  Unified  Web  Site  Accessibility  Guidelines  [On-line]. 
Available:  http ://w w w . w3  .org/W AI/GL/central . htm 

Collins,  S.  E.  (2001).  Web66:  International  School  Web  Site  Registry  [on-line].  Available: 
http://web66.coled.umn.edu/schools.html. 

Weeber,  J.E.  (1999).  What  could  I know  of  racism?  Journal  of  Counseling  and  Development,  77^  20-23. 

Weinrach,  S.G.,  & Thomas,  K.R.  (1998).  Diversity-sensitive  counseling  today:  A postmodern  clash  of 
values.  Journal  of  Counseling  and  Development,  76,  115-122. 

Wong,  M.  A.  (1997).  Disability  and  the  Internet:  Access  and  use  as  means  toward  greater  self-advocacy. 


58 


Physical  Disabilities:  Education  and  Related  Services,  15(2),  23-36. 

World  Wide  Web  Consortium  (1999).  Web  Accessibility  Initiative  (WAI):  HTML  Author  Guidelines— 
version  1.0  [On-line].  Available:  http://www.w3.org/WAI/GL/WD-WAI-HAG.html 

World  Wide  Web  Consortium  (2001a).  Web  Accessibility  Initiative  (WAI):  User  Agent  Accessibility  — 
version  1.0  [On-line].  Available:  http://www.w3.org/TR/2001/CR-UAAG10-20010912/ 

World  Wide  Web  Consortium  (2001b).  Web  Accessibility  Initiative  (WAI):  Web  Content  Accessibility 
Guidelines  2.0 -WiC  Working  Draft  [On-line].  Available:  http://www.w3.org/TR/WCAG20/ 


59 


An  Assessment  of  Students’  Preexisting  Computer  Skills  and  Attitudes 
Toward  Internet-Delivered  Instruction 


Jonathan  Brinkerhoff 
University  of  New  Mexico 

Carol  M.  Koroghlanian 
University  of  Wisconsin  - Eau  Claire 


Abstract 

Research  examining  the  rapid  expansion  of  Internet-delivered  instruction  has  by  and  large  reflected  rather 
than  driven  instructional  design.  Those  interested  in  maximizing  student  learning  and  motivation  have  a limited 
research  foundation  upon  which  to  base  instructional  strategies.  This  study  examined  students’  computer  skills, 
attitudes  toward  Internet-delivered  instruction,  and  the  instructional  components  students ' considered  important  in 
an  Internet  course  as  a function  of  demographics,  prior  computer  skills  and  Internet-delivered  course  experience. 
Results  suggest  that  student  attitudes  toward  Internet-delivered  instruction  are  fairly  positive,  with  students  having 
Internet-based  course  experience  more  likely  to  recommend  taking  an  online  course.  However,  overall  student 
computer  skill  levels  prerequisite  to  successful  participation  in  online  instruction  lack  depth.  Recommendations  for 
the  design  of  Internet  courses  and  future  research  are  discussed. 


Introduction 

Internet-based  information  and  communication  technologies  are  increasingly  being  used  for  delivery  or 
support  of  university  courses  (Hanna,  1998;  Johnson,  1999;  Mitra,  2000;  National  Center  for  Education  Statistics, 
1998;  Volery,  2001).  Such  uses  span  a wide  range  from  providing  online  support  for  traditional,  classroom-based 
courses  through  the  posting  of  syllabi,  lecture  notes  or  readings  on  the  World  Wide  Web  to  Internet-based  delivery 
of  entire  courses  incorporating  minimal  or  no  face-to-face  contact.  Much  of  the  research  examining  this  trend  has 
focused  on  student  achievement  (Windschitl,  1998;  Swan,  Shea,  Fredericksen,  Pickett,  Pelz  & Maher,  2000), 
descriptions  of  technology  issues  (Windschitl,  1998;  Mclsaac  & Gunawardina,  1996),  or  potential  cost  savings 
(Inglis,  1999).  Missing  from  the  published  research  is  significant  consideration  of  learner  characteristics  on  the 
design  of  Internet-delivered  courses. 

As  learners’  pre-existing  skills,  characteristics  and  attitudes  relevant  to  instructional  delivery  and  design 
can  impact  their  success  within  an  instructional  program  (Dick  & Carey,  1996;  Kemp,  Morrison  & Ross,  1998; 
Duggan,  Hess,  Morgan,  Kim  & Wilson,  1999),  learner  analysis  constitutes  a basic  component  in  most  instructional 
design  models.  Learner  analysis  components  important  as  a foundation  to  the  design  of  Internet-delivered 
instruction  include  learners’  technological  skills  as  well  as  their  attitudes  toward,  and  perceptions  of,  Internet- 
delivered  instruction.  Limited  studies  have  addressed  these  issues  (Duggan  et  ak,  1999;  Hara  & Kling,  1999;  Jiang 
& Ting,  1998),  yet  ignoring  learner  characteristics  can  reduce  the  effectiveness  of  Internet-delivered  courses 
(Wagner,  1993). 

Despite  limited  research  examining  learner  analysis  relevant  to  Internet  delivery  of  instruction,  instructional 
designers  may  find  the  extensive  body  of  research  investigating  computer  anxiety  and  attitudes  towards  technology 
applicable.  Instruments  developed  to  measure  computer  anxiety  include  Loyd  and  Gressard’s  Computer  Anxiety 
Scale  (CAS)  which  has  been  widely  used  for  adults  and  extensively  analyzed  by  psychometricians  for  validity  and 
reliability  (Loyd  & Gressard,  1984)  and  Delcourt  and  Kinzie’s  Attitudes  Toward  Computer  Technologies  (ACT) 
(Delcourt  & Kinzie,  1993).  Koroghlanian  and  Brinkerhoff  (2000-2001)  modified  items  from  the  CAS  and  ATC 
instruments  to  gather  information  regarding  learners’  attitudes  towards  and  perceptions  of  Internet-delivered  courses 
as  a means  for  informing  the  design  of  successful  Internet-delivered  courses.  The  present  study  extends  the  previous 
work  of  Koroghlanian  and  Brinkerhoff  (2000-2001)  by  addressing  limitations  in  the  earlier  survey  which  included 
the  limited  geographic  distribution  of  participants  as  well  as  the  low  number  of  participants  with  previous  Internet- 
delivered  instructional  experience. 

This  study  investigated  the  following  research  questions: 
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1.  What  is  the  current  state  of  computer  experience  and  skills  of  students? 

2.  Do  computer  experience  and  skills  differ  by  region? 

3.  What  components  do  students  deem  important  for  an  Internet-delivered  course?  Do  these  components 
differ  by  computer  experience,  demographics,  computer  skills  or  prior  Internet-delivered  course 
experience? 

4.  What  are  student  attitudes  towards  and  perceptions  of  Internet-delivered  courses?  Do  theses  attitudes 
and  perceptions  differ  by  demographics,  computer  experience,  computer  skills  or  prior  Internet- 
delivered  course  experience? 

Method 

Participants 

Five  hundred  twelve  students  from  six  universities  located  throughout  the  United  States  were  the 
participants  in  this  investigation.  Approximately  two  thirds  were  women,  44%  were  education  majors,  15%  were 
business  majors,  16%  were  humanities,  language  or  physical  education  majors  and  19%  were  educational 
technology,  engineering,  math,  science  or  computer  science  majors.  Of  the  277  participants  for  which  data  were 
available,  66%  were  undergraduates. 

In  terms  of  computer  experience,  91%  of  the  participants  had  used  a computer  four  or  more  years  and  67% 
used  a computer  six  or  more  hours  per  week.  In  addition,  73%  of  the  participants  had  taken  some  sort  of  computer 
course  during  their  college  careers.  Finally,  22%  of  the  participants  had  prior  Internet-delivered  course  experience. 

Data  collection  extended  from  November  of  2000  to  June  of  2001.  Three  hundred  forty-one  students 
enrolled  in  traditionally  delivered  classes  comp  leted  a paper-based  version  of  the  survey  distributed  and  collected 
during  normally  scheduled  class  time  while  an  additional  42  students  completed  an  online  version.  Students 
enrolled  in  Internet-delivered  classes  (129)  completed  the  online  version  of  the  survey. 

Criterion  Measures 

The  single  instrument  for  this  investigation  was  a survey.  The  survey  included  five  sections:  demographics 
and  computer  experience,  computer  skills  related  to  Internet  delivery  of  instruction,  attitudes  towards  and 
perceptions  of  Internet-delivered  courses,  rating  of  components  in  a Internet-delivered  course,  and  prior  distance 
education  experience.  Two  forms  of  the  survey  were  used,  a paper  based-version  and  an  electronic  version.  Both 
versions  included  identical  items  with  the  exception  of  the  demographics  section,  which  included  one  additional 
question  in  the  electronic  version  asking  participants  to  name  their  college  or  university. 

The  demographic  section  had  eight  forced  choice  items  covering  general  information  including  sex,  age, 
major,  years  of  computer  use,  hours  per  week  of  computer  use,  and  completion  of  college  computer  courses. 

The  computer  skills  section  incorporated  13  Likert  type  items  rated  on  a 5-point  scale  from  Completely 
Confident  to  Not  Confidence  at  All.  Participants  rated  their  perceived  computer  skills  for  each  item  covering  three 
broad  categories:  telecommunications,  technical  expertise,  and  Internet  browser  basics.  The  Chronbach  Alpha 
reliability  for  this  section  of  the  survey  was  0.92. 

The  attitudes  towards  Internet-delivered  courses  section  incorporated  19  Likert -type  items  on  a 5 -point 
scale  ranging  from  Strongly  Agree  to  Strongly  Disagree.  Many  of  the  items  were  based  on  items  from  the  CAS 
(Loyd  & Gressard,  1984)  and  the  ACT  measure  (Kinzie,  Delcourt,  & Powers,  1994).  The  Chronbach  Alpha 
reliability  for  this  section  was  0.87. 

The  survey  section  for  rating  of  components  in  an  Internet-delivered  course  incorporated  seven  Likert -type 
items  on  a 5-point  scale  ranging  from  Extremely  Important  to  Not  at  all  Important.  These  items  were  intended  to 
determine  which  features  participants  felt  were  important  components  to  an  Internet-delivered  course.  The 
Chronbach  Alpha  reliability  for  this  section  was  0.81. 

Only  subjects  with  distance  education  experience  completed  the  final  survey  section.  This  section 
consisted  of  12  forced  choice  items  and  six  Likert -type  items  on  a 5-point  scale  ranging  from  Strongly  Agree  to 
Strongly  Disagree.  The  12  forced  choice  items  elicited  details  about  participant’s  Internet-delivered  course 
experience  and  included  items  about  the  number  of  courses  taken,  the  inclusion  of  face-to-face  meetings,  the  type  of 
media  used  and  the  reason  for  taking  an  Internet-delivered  course.  The  six  Likert -type  items  solicited  participants’ 
feelings  toward  courses  delivered  via  the  Internet.  The  Chronbach  Alpha  reliability  for  this  section  was  0.70. 

Finally,  all  participants  were  asked,  “Would  you  be  willing  to  take  an  Internet-delivered  course?’’ 

Results 

Reported  results  for  this  investigation  are  separated  into  four  sections. 
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Computer  Skills  Related  to  Internet  Delivery  of  Instruction  - Means  and  standard  deviations  of  the  responses  for 
the  computer  skills  section  of  the  survey  are  reported  in  Table  1.  The  numbers  represent  responses  on  a 5-point 
Likert  scale  ranging  from  5 (completely  confident)  to  1 (not  confident  at  all).  Responses  are  separated  into  three 
categories:  participants  with  prior  Internet-delivered  course  experience  (N  = 1 16),  participants  with  no  prior 
experience  (N  = 386)  and  all  participants  (N  = 5 12). 

Participants  in  general  were  confident  in  their  skills  with  regards  to  email,  following  links  on  web  pages, 
and  sending  and  reading  attachments.  Participants  reported  less  confidence  concerning  use  of  FTP  programs, 
creating  web  pages,  and  locating  and  installing  plugins  (see  Table  1). 

There  were  large  differences  in  responses  between  participants  with  and  without  prior  Internet-delivered 
course  experience  for  several  items.  The  largest  differences  were  for  following  threaded  discussions,  installing  the 
latest  version  of  a browser,  locating  and  installing  plugins,  and  using  an  FTP  program. 

An  ANOVA  was  performed  comparing  means  for  the  computer  skills  section  of  the  survey  between 
participants  from  a large  Southwestern  university  (N  = 235)  and  a large  upper  Midwestern  university  (N  = 190). 
Significant  differences  in  perceived  computer  skills  between  participants’  responses  by  geographic  region  were 
found  for  1 1 of  thel3  items,  however  the  small  eta^  values  ranging  from  0.02  to  0.08  suggest  that  the  differences 
were  related  more  to  sample  size  than  any  significant  difference  in  computer  skills. 

Correlations  were  calculated  between  computer  skills  and  the  demographic  factors  of  age,  sex,  hours  per 
week  of  computer  use  and  years  of  computer  use.  Of  the  significant  correlations  found,  only  those  at  or  above  0.25 
are  reported  in  Table  2.  Age  and  hours  per  week  of  computer  use  were  the  only  results  with  correlations  greater 
than  0.25,  with  the  largest  correlations  between  hours  per  week  of  computer  use  and  opening  and  printing  files  with 
Acrobat,  locating  and  installing  plugins,  and  installing  the  latest  version  of  a browser. 


Table  1.  Computer  Skills  Responses  by  Prior  Internet-Delivered  Course  Experience 


Computer  Skills 

- 

Internet  Course  Experience 

Prior  No  Prior 

Experience  Experience 

(n=116)  (n  = 386) 

All 

Participants 
(n  = 512) 

Reading,  sending  and  deleting  email  messages 

M 

4.70 

4.71 

4.70 

0.65 

0.70 

.69 

Sending  and  reading  email  attachments 

M 

4.55 

4.19 

4.27 

0.88 

1.12 

1.08 

Subscribing  to  a listserv,  sending  a message  to  the  entire  listserv  and 

M 

3.76 

3.05 

3.21 

responding  to  just  the  person  who  posted  a message  to  a listserv 

SD 

1.42 

1.45 

1.47 

Following  threaded  discussions  and  posting  messages  on  bulletin  boards 

M 

3.99 

2.80 

3.07 

SD 

1.38 

1.48 

1.53 

Participating  in  chat  sessions 

M 

4.18 

3.40 

3.57 

SD 

1.22 

1.53 

1.50 

Installing  the  latest  version  of  a browser  such  as  Internet  Explorer  or 

M 

3.82 

2.95 

3.13 

Netscape 

SD 

1.51 

1.57 

1.60 

Locating  and  installing  plugins  such  as  Shockwave,  QuickTime  or  VRML 

M 

3.44 

2.51 

2.71 

SD 

1.65 

1.56 

1.63 

Following  links  on  web  pages  and  returning  to  the  starting  point 

M 

4.41 

4.42 

4.41 

SD 

1.11 

0.89 

0.94 

Recognizing  clickable  objects  on  web  pages 

M 

4.59 

4.52 

4.54 

SD 

0.92 

0.84 

0.86 

Downloading  files  embedded  on  web  pages 

M 

4.13 

3.55 

3.67 

SD 

1.23 

1.40 

1.38 

Creating  web  pages  with  images,  links  and  text 

M 

3.08 

2.45 

2.58 

SD 

1.63 

1.44 

1.50 

Opening  and  printing  files  with  Acrobat 

M 

3.78 

3.00 

3.17 

SD 

1.58 

1.56 

1.60 

Using  an  FTP  program  to  upload  or  download  files 

M 

3.18 

2.26 

2.47 
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SD  1.60  1.41  1.50 

Note.  Responses  ranged  from  5 (Completely  Confident)  to  1 (Not  Confident  at  all).  Ten  participants  did  not  indicate  whether  they 
had  previous  distance  education  experience. 


Attitudes  Towards  and  Perceptions  of  Internet-Delivered  Courses  - Means  and  standard  deviations  of  the 
responses  for  the  attitudes  section  of  the  survey  are  reported  in  Table  3.  The  numbers  represent  responses  on  a 5- 
point  Likert  scale  ranging  from  5 (strongly  agree)  to  1 (strongly  disagree).  While  statements  appear  in  their  original 
form,  results  for  negatively  stated  items  were  reverse  coded  for  the  purposes  of  statistical  analysis.  Results  of 
attitudes  towards  Internet-delivered  courses  by  prior  experience  indicate  that  participants  in  general  were  slightly 
positive  towards  most  items.  Two  items  had  large  differences  in  responses  between  participants  with  and  without 
prior  Internet-delivered  course  experience.  The  largest  difference  was  for  the  item  concerning  choosing  an  Internet 
section  of  a course,  while  the  second  concerned  being  upset  if  a required  course  were  only  offered  over  the  Internet. 

An  ANOVA  was  performed  comparing  means  for  the  attitudes  section  of  the  survey  between  participants 
from  the  Southwest  and  upper  Midwest.  Significant  differences  between  participants’  responses  by  geographic 
region  were  found  for  six  of  the  19  items.  Again,  small  eta^  values  ranging  from  0.02  to  0.03  suggest  that  the 
differences  were  related  more  to  sample  size  than  any  significant  difference  in  attitudes. 

Table  2.  Correlations  of  Demographic  Data  with  Computer  Skills 


Hours  Per  Week  of 
Computer  Use 

Age 

Computer  Skills 

Correlation 

a 

Correlation 

a 

Reading,  sending  and  deleting  email  messages 

NA 

NS 

NA 

NS 

Sending  and  reading  email  attachments 

0.313 

< 0.001 

NA 

NS 

Subscribing  to  a listserv 

0.314 

< 0.001 

NA 

NS 

Following  threaded  discussions 

0.388 

< 0.001 

0.278 

< 0.001 

Participating  in  chat  sessions 

0.341 

< 0.001 

NA 

NS 

Installing  the  latest  version  of  a browser 

0.441 

< 0.001 

0.280 

< 0.001 

Locating  and  installing  plugins 

0.458 

< 0.001 

0.286 

< 0.001 

Following  links  on  web  pages 

0.249 

< 0.001 

NA 

NS 

Recognizing  clickable  objects 

0.235 

< 0.001 

NA 

NS 

Downloading  files  embedded  on  web  pages 

0.410 

< 0.001 

NA 

NS 

Creating  web  pages 

0.404 

< 0.001 

0.277 

< 0.001 

Opening  and  printing  files  with  Acrobat 

0.466 

< 0.001 

0.245 

< 0.001 

Using  an  FTP  program 

0.412 

< 0.001 

0.331 

< 0.001 

Note.  NA  = Non-applicable,  correlation  was  less  than  0.25.  NS  = Non-significant  result. 


Correlations  between  demographic  data  and  attitudes  were  calculated  and  revealed  numerous  significant 
correlations,  with  those  greater  than  0.25  reported  in  Table  4.  Most  of  these  correlations  were  between  hours  per 
week  of  computer  use  and  attitudes  and  between  age  and  attitudes.  The  largest  correlations  for  hours  per  week  of 
computer  use  were  for:  (1)  doing  well  with  computers,  (2)  convenience  of  taking  Internet  courses,  and  (3)  having 
access  to  Internet  courses.  The  largest  correlations  for  age  were  for:  (1)  the  impersonal  nature  of  Internet  courses, 
(2)  convenience  of  taking  Internet  courses,  and  (3)  feeling  upset  with  required  Internet  courses.  Several  significant 
correlations  were  found  between  years  of  computer  use  and  attitudes,  however  only  one  was  greater  than  .25:  doing 
well  in  an  Internet  course. 

Rating  of  Components  in  an  Internet-Delivered  Course  - Means  and  standard  deviations  of  the  responses  for  the 
rating  of  components  section  of  the  survey  are  reported  in  Table  5.  The  numbers  represent  responses  on  a 5-point 
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Likert  scale  ranging  from  5 (extremely  important)  to  1 (not  at  all  important).  Overall,  participants  felt  that  email 
communication  with  the  instructor,  telephone  technical  support  and  instructions  for  installing  and  configuring 
software  represented  the  most  important  components  in  an  Internet-delivered  course.  Participants  felt  the  least 
important  component  was  a weekly  online  chat  time.  Participants  with  and  without  prior  experience  rated  the 
components  quite  similarly.  The  largest  differences  concerned  having  a weekly  chat  time  (M  = 3.33  for  prior 
experience,  M = 3.99  for  no  prior  experience)  and  having  an  initial  face-to-face  meeting  at  the  beginning  of  the 
course  (M  = 3.61  for  prior  experience,  M = 4. 17  for  no  prior  experience). 

An  ANOVA  performed  on  each  of  the  items  revealed  no  significant  differences  for  participants  by  geographic 
region. 

Correlations  between  demographic  data  and  components  were  calculated.  These  correlations  were  not 
numerous  and  small  to  moderate  in  magnitude.  The  largest  correlation  was  between  age  and  having  an  initial  face- 
to-face  meeting,  correlation  = 0.259. 

Prior  Internet-Delivered  Course  Experience  - Of  the  512  participants  in  this  survey,  116  (23%)  had  experienced 
some  sort  of  Internet-delivered  course  prior  to  the  time  of  this  survey.  Table  6 reports  the  types  of  media  these 
participants  used  in  their  Internet-delivered 


Table  3.  Attitudes  Towards  Internet -Delivered  Courses  by  Prior  Internet-Delivered  Course  Experience 


Internet  Course  Experience 


Attitudes 

Prior 

Experience 
(n=  116) 

No  Prior 
Experience 
(n  = 386) 

All 

Participants 
(n  = 512) 

I could  do  well  in  an  Internet -delivered  course 

M 

4.44 

3.89 

4.02 

s 

r» 

0.94 

1.01 

1.02 

I don’t  do  well  with  computers* 

u 

M 

4.43 

4.07 

4.16 

0.96 

1.06 

1.04 

Taking  an  Internet-delivered  course  would  be  one  way  to  stay  current 

M 

4.33 

3.91 

4.00 

with  new  technology 

SD 

0.80 

0.96 

0.94 

Internet -delivered  courses  are  an  efficient  way  to  leam 

M 

3.86 

3.32 

3.45 

1.13 

1.04 

1.08 

I would  be  upset  if  a required  course  was  only  offered  over  the  Internet* 

M 

3.41 

2.49 

2.70 

SD 

1.47 

1.28 

1.38 

I feel  at  ease  using  the  web 

M 

4.51 

4.31 

4.36 

0.92 

0.76 

0.80 

Problems  with  using  technology  get  in  the  way  of  learning  from  the 

M 

2.87 

3.05 

3.01 

Internet* 

1.32 

1.09 

1.15 

If  I had  a choice  between  taking  an  Internet  section  or  a classroom  section 

M 

3.43 

2.41 

2.64 

of  the  same  course,  I would  choose  the  Internet  section 

SD 

1.41 

1.17 

1.30 

If  I took  a course  delivered  over  the  Internet,  I would  feel  isolated* 

M 

3.44 

2.86 

3.00 

1.29 

1.19 

1.23 

Internet -delivered  courses  provide  a greater  opportunity  for  interactivity 

M 

2.54 

2.35 

2.39 

between  students  and  between  students  and  instructor 

SD 

1.19 

1.01 

1.06 

I feel  the  Internet  is  as  informative  as  a teacher 

M 

2.77 

2.60 

2.64 

1.23 

1.13 

1.16 

It  would  be  exciting  to  take  a course  delivered  over  the  Internet 

M 

3.80 

3.12 

3.28 

SD 

1.09 

1.10 

1.13 

If  I took  Internet -delivered  courses,  it  would  be  a chance  to  leam  about 

M 

3.63 

3.48 

3.51 

the  Internet 

SD 

1.14 

1.02 

1.05 
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I feel  that  Internet -delivered  courses  are  impersonal* 

M 

2.97 

2.31 

2.47 

1.15 

1.01 

1.08 

I don’t  have  access  to  equipment  that  would  allow  me  to  take  an  Internet- 

M 

4.46 

3.92 

4.05 

delivered  course* 

0.94 

1.23 

1.19 

I dislike  using  the  Internet  for  educational  purposes* 

M 

4.28 

3.84 

3.94 

1.05 

1.06 

1.07 

Taking  an  Internet-delivered  course  would  be  a good  way  to  learn 

M 

3.91 

3.27 

3.41 

0.97 

0.97 

1.01 

It  would  be  convenient  for  me  to  take  an  Internet -delivered  course 

M 

4.14 

3.46 

3.62 

SD 

1.09 

1.14 

1.16 

I would  be  anxious  about  taking  an  Internet -delivered  course* 

M 

3.43 

2.85 

2.98 

1.37 

1.12 

1.20 

Note.  Responses  ranged  from  5 (Strongly  Agree!  to  1 (Strongly  Disagree!.  Ten  narticinants  failed  to  indicate  whether  they  had 

previous  distance  education  experience.  Asterisked  items  (*)  were  reverse  coded  and  responses  in 

the  table  reflect  this 

reverse 

coding. 

Table  4.  Correlations  of  Demographic  Data  with  Attitudes 

Hours  Per  Week 

Age 

Years  of 

of  Computer  Use 

Computer  Use 

Attitudes 

Corr. 

a 

Corr. 

a 

Corr. 

a 

I could  do  well  in  an  Internet-delivered  course 

0.268 

< 0.001 

NA 

NS 

0.263 

< 0.001 

I don’t  do  well  with  computers* 

0.380 

< 0.001 

NA 

NS 

NA 

NS 

Taking  an  Internet-delivered  course  would  be  one  way  to  stay 
current  with  new  technology 

NA 

NS 

NA 

NS 

NA 

NS 

Internet -delivered  courses  are  an  efficient  way  to  learn 

NA 

NS 

0.262 

< 0.001 

NA 

NS 

I would  be  upset  if  a required  course  was  only  offered  over  the 
Internet* 

0.278 

< 0.001 

NA 

NS 

0.267 

< 0.001 

I feel  at  ease  using  the  web 

NA 

NS 

NA 

NS 

NA 

NS 

Problems  with  using  technology  get  in  the  way  of  learning  from 
the  Internet* 

NA 

NS 

NA 

NS 

NA 

NS 

If  I had  a choice  between  taking  an  Internet  section  or  a 
classroom  section  of  the  same  course,  I would  choose  the  Internet 
section 

0.255 

< 0.001 

0.261 

< 0.001 

NA 

NS 

If  I took  a course  delivered  over  the  Internet,  I would  feel 
isolated* 

NA 

NS 

NA 

NS 

NA 

NS 

Internet -delivered  courses  provide  a greater  opportunity  for 
interactivity  between  students  and  between  students  and 
instructor 

NA 

NS 

NA 

NS 

NA 

NS 

I feel  the  Internet  is  as  informative  as  a teacher 

NA 

NS 

NA 

NS 

NA 

NS 

It  would  be  exciting  to  take  a course  delivered  over  the  Internet 

0.270 

< 0.001 

NA 

NS 

NA 

NS 

If  I took  Internet -delivered  courses,  it  would  be  a chance  to  learn 
about  the  Internet 

NA 

NS 

NA 

NS 

NA 

NS 

I feel  that  Internet -delivered  courses  are  impersonal* 

NA 

NS 

0.314 

< 0.001 

NA 

NS 

I don’t  have  access  to  equipment  that  would  allow  me  to  take  an 
Internet -delivered  course* 

0.286 

< 0.001 

NA 

NS 

NA 

NS 

I dislike  using  the  Internet  for  educational  purposes* 

0.270 

< 0.001 

NA 

NS 

NA 

NS 
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Taking  an  Internet -delivered  course  would  be  a good  way  to 
learn 

It  would  be  convenient  for  me  to  take  an  Internet -delivered 
course 

I would  be  anxious  about  taking  an  Internet -delivered  course* 


NA 

NS 

NA 

NS 

NA 

NS 

0.307 

< 0.001 

0.289 

< 0.001 

NA 

NS 

NA 

NS 

NA 

NS 

NA 

NS 

Note.  Attitudes  marked  with  an  * were  reverse  coded.  NA  = Non-applicable,  correlation  was  less  than  0.25.  NS  = Non- 
significant result. 


courses.  The  most  common  media  used  were  email,  web  site  and  electronic  bulletin  boards;  by  far  the  least  used 
medium  was  MOO/MUD.  Furthermore,  46%  of  the  participants  reported  that  the  course(s)  they  took  met  face-to- 
face  at  least  once  and  81%  reported  that  sufficient  technical  assistance  was  available  during  the  course(s). 

In  responding  to  why  these  participants  took  an  Internet-delivered  course,  14%  cited  curiosity,  60%  cited 
convenience,  22%  indicated  that  no  comparable  course  was  available  locally,  15%  indicated  the  course  was 
required,  30%  indicated  they  were  working  on  an  online  degree,  and  12%  cited  other  reasons.  In  addition,  81%  of 
these  participants  felt  they  had  received  sufficient  technical  support,  90%  rated  their  experience  as  good  to  excellent 
and  90%  would  recommend  taking  an  Internet-delivered  course  to  a friend. 

Table  5.  Rating  of  Components  by  Prior  Internet -Delivered  Course  Experience 


Distance  Education  Experience 


Components 

Prior  Experience 
(n=  116) 

No  Prior  Experience 
(n  = 386) 

All  Participants 
(n  = 502) 

Step-by-step  instmctions  and  guides  for  using, 
installing  and  configuring  software  are  posted  on  the 
web  site 

M 

SD 

4.16 

1.18 

4.28 

0.94 

4.25 

1.00 

A face-to-face  meeting  at  the  beginning  of  the  course 

to  provide  orientation  to  the  software,  install. 

M 

3.61 

4.17 

4.04 

configuration  and  usage;  attendance  would  be  optional 

1.46 

1.00 

1.14 

Telephone  technical  support  for  software  and  hardware 

M 

4.23 

4.39 

4.35 

problems 

1.04 

0.82 

0.87 

A scheduled  weekly  “chat  time”  when  the  entire  class 

M 

3.33 

3.99 

3.84 

and  instmctor  are  online  for  discussions  and  questions 

1.52 

1.13 

1.26 

Incorporating  a listserv  or  electronic  bulletin  board  to 

M 

4.25 

4.13 

4.16 

distribute  general  announcements,  conduct  discussions 

1.09 

0.98 

1.01 

and  ask  questions 

Email  communication  with  the  instructor 

M 

4.76 

4.61 

4.65 

SD 

0.61 

0.75 

0.72 

Email  with  individual  students  in  the  class 

M 

4.22 

4.10 

4.13 

SD 

1.04 

0.97 

0.99 

Note.  Responses  ranged  from  5 (Extremely  Important)  to  1 (Not  at  all  Important).  Ten  participants  did  not  indicate  whether  they 
had  previous  distance  education  experience. 


These  participants  rated  various  aspects  of  their  Internet-delivered  course  experience  and  the  results  are 
reported  in  Table  7.  One  item  concerning  isolation  was  stated  negatively  and  the  results  were  reverse  coded  for 
purposes  of  analysis.  Overall  responses  were  fairly  positive.  Not  having  to  meet  at  a specific  place  and  time  and 
enjoying  working  at  their  own  pace  were  rated  the  most  positively,  while  the  statement  concerning  local  availability 
was  rated  lowest. 

Finally,  all  512  participants  were  asked,  “Would  you  be  willing  to  take  an  Internet-delivered  course?”  Of 
the  500  participants  responding  to  this  question,  394  (79%)  said  yes.  Looking  at  participants  with  prior  distance 
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education  experience,  89%  responded  yes  to  this  question.  The  difference  between  participants  with  and  without 
prior  Internet-delivered  course  experience  was  significant,  F (1,  499)  = 7.538,  p = .006,  eta^  = .02. 


Table  6.  Media  Components  used  in  Internet-Delivered  Courses 


Media  Components 

Used 

Not  Used 

Print 

76.6% 

15.9% 

Email 

92.6% 

4.6% 

Listserv 

47.7% 

36.4% 

Chat  Room 

53.3% 

38.3% 

MOO  or  MUD 

9.5% 

61.0% 

Web  Site 

87.0% 

10.2% 

Electronic  Bulletin  Boards 

82.2% 

15.0% 

PDF  files  / Acrobat  Reader 

52.3% 

35.5% 

Note.  N = 107. 

Table  7.  Internet-Delivered  Course  Experience 


Mean 

Standard 

Deviation 

I appreciated  being  able  to  work  at  my  own  pace 

4.52 

0.76 

Not  having  to  meet  at  a specific  place  and  time  was  convenient 

4.60 

0.78 

I received  instmction  I couldn’t  have  gotten  locally 

3.32 

1.32 

I felt  there  was  enough  interaction  between  students 

3.51 

1.16 

I felt  there  was  enough  interaction  between  students  and  instructor 

3.56 

1.26 

I felt  I was  working  in  isolation* 

3.42 

1.41 

Note.  N = 104.  Responses  ranged  from  5 (Strongly  Agree)  to  1 (Strongly  Disagree).  The  asterisked  (*)  statement  was  reverse 
coded  and  the  results  in  the  table  reflect  the  reverse  coding. 


Discussion 

The  purposes  of  this  study  were  to  (1)  survey  a widely  diverse  sample  of  college  students’  existing 
computer  skills  and  attitudes  towards  and  perceptions  of  Internet-delivered  courses  and  (2)  use  those  results  to 
formulate  recommendations  for  the  design  of  Internet-delivered  courses. 

Computer  Skills  Related  to  Internet  Delivery  of  Instruction 

Participants  in  this  survey  rate  themselves  as  proficient  web  surfers  and  users  of  email  and  email 
attachments.  They  rate  themselves  as  less  capable  with  more  technical  skills  such  as  installing  browsers  and 
plugins,  web  page  creation  or  use  of  an  FTP  program. 

Substantial,  broadly -based  differences  in  perceived  skills  between  participants  with  and  without  Internet- 
delivered  course  experience  were  revealed.  In  particular,  those  with  prior  experience  reported  greater  familiarity 
with  following  threaded  discussions,  installing  browsers  and  associated  plugins,  and  use  of  FTP  programs.  These 
results  suggest  that  increased  skill  levels  result  from  increased  experience,  a supposition  supported  by  the 
correlational  data  in  Table  2 indicating  that  more  hours  per  week  of  computer  use  translate  into  higher  skill  levels. 
Years  of  computer  use  also  reflect  increased  experience,  and  multiple  correlations  with  skills  were  found  but  the 
correlations  were  lower  than  those  for  hours  per  week  of  computer  use.  Correlations  between  age  and  computer 
skills  provide  additional  evidence  for  supposing  that  increased  skills  result  fro  m increased  experience,  as  younger 
participants  generally  reported  higher  levels  of  computer  use. 
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These  results  suggest  that  the  increasing  use  of  email  and  web  surfing  have  provided  many  users  with  a 
baseline  level  of  competence,  however,  intermediate  to  advanced  skills  are  lacking.  The  shallow  level  of  computer 
skills  may  result  in  significant  challenges  for  students  enrolled  in  Internet-delivered  courses  as  well  as  their 
designers.  For  example,  designers  intent  on  providing  more  dynamic  or  interactive  learning  environments  requiring 
installation  of  a browser  plugin  could  be  inadvertently  placing  landmines  of  potential  frustration  in  the  path  of 
students.  This  is  supported  by  results  of  a survey  of  206  online  MBA  students  (Schramm  et  al.,  2000)  which  found 
students  quickly  became  frustrated  with  technology  that  failed  to  work  or  wasn’t  user  friendly.  On  a more  positive 
note,  these  results  also  suggest  that  the  experience  of  completing  Internet-delivered  courses  supports  the  acquisition 
of  increased  computer  skills. 

Results  comparing  students  from  the  Southwest  and  upper  Midwest  imply  that  levels  of  computer  skill  are 
relatively  evenly  distributed  geographically.  While  statistically  significant  differences  in  reported  computer  skills 
were  found,  the  effect  sizes  ranging  from  .02  to  .08  were  small.  Similarities  in  reported  patterns  of  computer  use 
also  suggest  an  even  distribution  of  computer  skills.  ANOVAs  comparing  both  groups’  hours  per  week  of  computer 
use  and  years  of  computer  use  revealed  significant  differences  for  both  but  effect  sizes  of  0.01  and  0.02  respectively. 
The  small  effect  sizes  suggest  that  any  significant  differences  were  more  a result  of  sample  size  than  any  meaningful 
difference  in  skill  levels.  This  implies  that  instructional  designers  are  likely  to  find  consistent  patterns  of  computer 
skills  in  different  geographic  regions. 

Additionally,  both  groups  reported  similar  completion  rates  for  computer  literacy  courses,  with  73  percent 
of  Southwest  students  and  69  percent  of  upper  Midwest  students  having  taken  such  a course.  The  shallow  skill 
levels  reported  by  participants  suggests  that  completion  of  such  courses  does  little  to  promote  skills  relevant  to 
successful  participation  in  Internet-delivered  instruction. 

Attitudes  Towards  and  Perceptions  of  Internet-Delivered  Courses 

Overall,  participants  were  neutral  to  slightly  positive  towards  Internet-delivered  instruction.  They  felt  at 
ease  using  the  Web  and  agreed  that  taking  an  Internet-delivered  course  would  help  them  stay  current  with  new 
technology.  In  contrast,  participants  were  neutral  to  slightly  negative  concerning  the  interactivity  of  Internet- 
delivered  courses  and  felt  that  such  courses  were  impersonal. 

Large  differences  exist  between  participants  with  and  without  prior  Internet-delivered  course  experience 
concerning  the  desirability  of  such  courses.  Participants  without  prior  distance  education  experience  are  much  less 
enthusiastic  about  choosing  an  Internet-delivered  section  of  a course,  would  more  likely  be  upset  if  such  a course 
were  required,  and  are  less  likely  to  view  such  courses  as  a good  way  to  learn.  These  feelings  may  be  based  on  their 
greater  perception  that  Internet-delivered  courses  are  impersonal  and  not  particularly  convenient.  The  large 
differences  in  attitudes  between  those  with  and  without  prior  Internet  course  experience  suggest  that  participation  in 
such  courses  contributes  toward  reducing  these  negative  perceptions. 

Results  comparing  students  from  the  Southwest  and  upper  Midwest  suggest  that  attitudes  toward  Internet- 
delivered  instruction  are  largely  consistent  geographically.  Effect  sizes  ranged  from  .02  to  .03  for  all  statistically 
significant  differences  again  suggesting  that  these  differences  were  related  to  sample  size. 

Correlation  results  indicated  that  students  spending  more  hours  per  week  working  with  computers  generally 
held  more  positive  attitudes,  while  older  students  were  generally  more  negative.  Designers  might  consider  limiting, 
at  least  initially,  the  technical  requirements  of  courses  aimed  at  older  learners  while  providing  greater  levels  of 
support. 

Rating  of  Components  in  an  Internet-Delivered  Course 

This  part  of  the  survey  investigated  what  components  students  value  in  Internet-delivered  courses.  These 
findings  clearly  indicate  students  value  email  communication  with  the  instructor.  Also  receiving  high  rankings  were 
telephone  technical  support  and  step-by-step  instructions  for  installing  and  configuring  software.  Receiving  a more 
neutral  rating  was  a weekly  scheduled  “chat  time’’  for  students  and  instructor. 

Responses  for  those  with  and  without  prior  Internet-delivered  course  experience  were  quite  similar  for  most 
components,  with  the  largest  differences  occurring  for  a weekly  “chat  time”  and  meeting  face-to-face  at  the 
beginning  of  the  course.  Those  with  prior  experience  placed  less  value  on  both  these  components. 

Limited  correlations  of  small  magnitude  were  found  for  this  section  of  the  survey  with  the  exception  of  a correlation 
of  0.259  between  age  and  having  an  initial  face-to-face  meeting. 

Implementation  of  several  components  identified  by  students  as  desirable  could  require  expenditures  of 
time  and  money  that  might  exceed  those  available  for  the  design  and  delivery  of  many  courses.  Failing  to  meet 
students’  perceived  needs  in  the  implementation  of  Internet-delivered  classes  might  lead  to  reduced  levels  of 
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satisfaction  and  support.  Before  committing  to  Internet-delivered  instruction,  instructors  and  institutions  might 
consider  whether  resources  are  available  to  provide  Internet  courses  likely  to  meet  students’  expectations. 

Prior  Internet- Delivered  Experience 

Participants  with  prior  Internet-delivered  course  experience  comprised  22%  of  respondents.  As  a group, 
they  were  positive  toward  their  online  experience,  particulaily  appreciating  the  convenience  and  ability  to  work  at 
their  own  pace  that  Internet-delivered  instruction  provided.  Fully  90%  would  recommend  taking  an  Internet- 
delivered  course  to  a friend. 

Participants  with  prior  Internet-delivered  course  experience  identified  email  and  web  sites  as  the  two  most 
common  components  used  in  the  Internet  courses  they  had  completed.  As  the  results  for  all  participants  identified 
use  of  email  and  surfing  the  web  as  the  two  computer  skills  they  felt  most  self-assured  about,  many  students  may 
possess  basic  skills  required  of  Internet  courses.  However,  electronic  bulletin  boards  represented  the  next  most 
commonly  used  component,  and  results  for  all  participants  identified  use  of  bulletin  boards  as  one  of  the  skills  about 
which  they  felt  least  secure. 

Suggestions  for  Course  Design  and  Implementation 

1.  Create  a database  of  user  friendly,  instructionally  sound,  frequently  revised  scaffolds  addressing  technical 
issues  relevant  to  Internet-delivered  instruction  such  as  browser  and  plugin  installation,  setting  printers, 
editing  preferences,  using  Acrobat  files,  etc.  Such  scaffolds  could  be  made  available  to  the  entire  campus 
through  handouts  at  computer  labs  or  online  through  university  websites.  Standardizing  and  sharing  such 
scaffolds  on  an  institution-wide  bases  would  reduce  individual  teacher  workloads  in  trying  to  support  their 
students. 

2.  Provide  technical  support  seven  days  a week  for  extended  hours  to  assist  with  general  hardware  and 
software  problems.  Unique  problems  relevant  to  an  individual  course  would  be  handled  by  the  faculty. 

3.  Design  Internet-delivered  courses  to  accommodate  a wide  range  of  student  computer  skills  to  avoid 
overwhelming  low-skill  students. 

4.  Perform  a task  analysis  for  each  Internet-delivered  course  to  identify  prerequisite  technical  skills  and 
include  this  information  in  course  descriptions. 

5.  Accommodate  older  students  by  providing  an  initial  face-to-face  meeting,  limiting  initial  use  of  more 
demanding  technologies,  and  offering  more  layers  of  support. 

6.  If  applicable,  examine  the  curricula  of  your  university’s  computer  literacy  course  to  identify  missing  skills 
necessary  for  supporting  Internet-delivered  instruction.  The  curricula  of  the  course  could  be  expanded  to 
cover  such  skills  and  the  course  could  then  be  listed  as  a prerequisite  for  enrollment  in  Internet  courses. 
Conversely,  the  design  of  Internet  courses  could  be  constrained  to  align  with  the  covered  skills. 

7.  Conduct  interviews  and/or  surveys  by  neutral  parties  to  determine  overall  Internet-delivered  course 
satisfaction.  Include  any  students  who  dropped  out  of  the  course. 


Suggestions  for  Future  Research 

New  technologies  continually  expand  the  options  for  design  of  instruction,  while  student’s  computer  skills  and 
attitudes  change  over  time.  Student  surveys  should  be  performed  on  an  on-going  bases  to  ensure  a satisfactory 
match  between  learners  and  course  design. 
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Introduction 

According  to  Information  Power  (1998)  collaboration  between  library  media  specialists  and  different 
members  of  the  learning  community  is  key  to  building  partnerships  within  the  school.  Of  prime  importance  are 
those  between  the  media  specialist  and  various  teachers.  School  health  education  teachers  have  a natural,  largely 
untapped,  alliance  with  library  media  specialists.  Until  now,  there  has  been  little  dis  cussion  of  the  interdisciplinary 
relationships  between  health  teachers  and  library  media  specialists.  There  are  great  opportunities  for  these  tow 
groups  to  work  closely  in  planning,  implementing  and  evaluating  health  lessons,  projects  and  research.  The 
National  Health  Education  Standards  focus  on  the  development  in  students  of  health  literacy:  this  notion  is 
operationalized  through  adolescent  development  of  skills  in  accessing  valid  health  information  and  services, 
analyzing  the  influence  of  media,  culture  and  technology  on  health;  and  advocating  for  personal,  family  and 
community  health,  among  others.  Library  media  specialists,  as  emphasized  in  the  “Information  Literacy  Standards 
for  Student  Learning’’  (Information  Power,  p.  8)  are  ideally  suited  to  collaboration  with  health  teachers  in  providing 
students  with  resources,  technological  expertise,  and  media  critiques,  as  well  as  advocacy  competencies,  etc.  This 
paper  will  address  the  library  media  specialist  in  collaboration  with  the  health  educator  in  terms  of  providing 
information  and  materials  to  health  education  students  in  the  K-12  educational  environment. 


The  Library  Media  Specialists 

Information  Power  (1998).  which  includes  national  guidelines  for  school  library  media  specialists,  is 
prepared  by  two  prominent  professional  organizations  to  which  many  school  library  media  specialists  belong:  The 
American  Association  of  School  Librarians  (AASL)  and  the  Association  for  Educational  Communications  and 
Technology  (AECT).  A major  theme  in  these  guidelines  is  that  of  building  partnerships  for  learning.  This  includes 
nine  information  literacy  standards  for  student  learning  and  focuses  on  creating  a community  of  lifelong  learners. 
The  information  literacy  standards  for  student  learning  are  divided  into  three  divisions:  information  literacy, 
independent  learning,  and  social  responsibility.  The  following  table  presents  these  standards  with  the  main 
indicators  for  each  standard.  Highlighted  areas  show  relationship  to  health  education  standards. 


Information  Literacy  Standards  for  Student  Learning 


Information  Literacy  Standards 

Independent  Learning  Standards 

Social  Responsibility  Standards 

Standard  1 : The  student  who  is 
information  literate  accesses 
information 

efficiently  and  effectively 

• recognizes  the  need  for 
information 

• recognizes  that  accurate  and 
comprehensive  information  is 
the  basis  for  intelligent  decision 
making 

• formulates  questions  based  on 
information  needs 

Standard  4:  The  student  who  is  an 
independent  learner  is  information 
literate  and  pursues  information 
related  to  personal  interests. 

• seeks  information  related  to 
various  dimensions  of  personal 
well-being,  such  as  career 
interests,  community 
involvement,  health  matters,  and 
recreational  pursuits 

• designs,  develops,  and  evaluates 

Standard  7:  The  student  who 
contributes  positively  to  the  learning 
community  and  to  society  is 
information  literate  and  recognizes 
the  importance  of  information  to  a 
democratic  society. 

• seeks  information  from 
diverse  sources,  contexts, 
disciplines,  and  cultures 

• respects  the  principle  of 
equitable  access  to  information 
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• 

identifies  a variety  of  potential 

information  products  and 

sources  of  information 

solutions  related  to  personal 

• 

develops  and  uses  successful 
strategies  for  locating 
information 

interests 

Standard  2:  The  student  who  is 

Standard  5:  The  student  who  is  an 

Standard  8:  The  student  who 

information  literate  evaluates 

independent  learner  is  information 

contributes  positively  to  the  learning 

information  critically  and 

literate  and  appreciates  literature  and 

community  and  to  society  is 

competently. 

other  creative  expressions  of 

information  literate  and  practices 

information. 

ethical  behavior  in  regard  to 

• 

determines  accuracy,  relevance. 

information  and  information 

and  comprehensiveness 

• is  a competent  and  self- 

technology . 

• 

distinguishes  among  fact,  point 

motivated  reader 

of  view,  and  opinion 

• derives  meaning  from 

• respects  the  principles  of 

• 

identifies  inaccurate  and 

information  presented  creatively 

intellectual  freedom 

misleading  information 

in  a variety  of  formats 

• respects  intellectual  property 

• 

selects  information  appropriate 

• develops  creative  products  in  a 

rights 

to  the  problem  or  question  at 

variety  of  formats 

• uses  information  technology 

hand. 

responsibly 

Standard  3:  The  student  who  is 

Standard  6:  The  student  who  is  an 

Standard  9:  The  student  who 

information  literate  uses  information 

independent  learner  is  information 

contributes  positively  to  the  learning 

accurately  and  creatively . 

literate  and  strives  for  excellence  in 

community  and  to  society  is 

information  seeking  and  knowledge 

information  literate  and  participates 

• 

Organizes  information  for 

generation. 

effectively  in  groups  to  pursue  and 

practical  application 

generate  information. 

• 

Integrates  new  information  into 

• assesses  the  quality  of  the 

one’s  own  knowledge 

process  and  products  of 

• shares  knowledge  and 

• 

Applies  information  in  critical 

personal  information  seeking 

information  with  others 

thinking  and  problem  solving 

• respects  other’s  ideas  and 

• 

Produces  and  commu  nicates 

• devises  strategies  for 

backgrounds  and  acknowledges 

information  and  ideas  in 

revising,  improving,  and 

their  contributions 

appropriate  formats 

updating  self-generated 

• and  through  technologies,  to 

knowledge 

design,  develop,  and  evaluate 
information  products  and 
solutions 

(Information  Power,  p.  8-43) 


The  Health  Educators 

Health  educators  and  pre -service  health  majors  have  been  challenged  to  conceptualize  and  develop 
comprehensive  school  health  education  programs  by  following  the  National  Health  Education  Standards  (NHES)  — 
a conceptual  framework  adopted  through  a coalition  of  professional  health  associations  — including  the  American 
School  Health  Association,  American  Public  Health  Association,  and  the  American  Cancer  Society.  These  P- 12 
standards  describe  skills  necessary  for  adolescents  to  become  “health  literate.”  Thus,  health  educators  are 
challenged  to  design  and  learn  to  coordinate  school  education  and  service  programs  across  components  (health 
services,  health  environment,  etc.)  that  maximize  the  potential  for  their  adolescent  health  students  to  acquire  skills  in 
accessing  appropriate  health  information  and  services;  analyzing  media,  culture  and  technology  for  their  impact  on 
health;  and  advocating  for  personal  family  and  community  health,  among  others.  Table  Two  below  includes  the 
relevant  standards.  Highlighted  areas  show  relationship  to  school  library  media  standards. 
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National  Health  Education  Standards  (K-12). 


Students  will. . . 

1.  Comprehend  concepts  of  health  promotion  and  disease  prevention. 

2.  Demonstrate  the  ability  to  ACCESS  valid  health  information  & services 

3.  Practice  health-enhancing  & health-reducing  behaviors 

4.  Analyze  the  influence  of  culture,  media,  technology  & other  factors  on  health. 

5.  Demonstrate  ability  to  use  interpersonal  communication  skills 

6.  Demonstrate  ability  to  use  goal-setting  & decision-making  skills 

7.  Demonstrate  ability  to  advocate  for  personal,  family,  & community  health 

(National  Health  Education  Standards  [K-1]) 


Library  Media  Specialists  and  Health  Educators  in  Collaboration 

The  tables  above  illustrate  commonalities  between  the  disciplines  of  the  library  media  specialist  and  the 
health  educator,  and  point  to  the  ever-increasing  necessity  to  teach  young  people  how  to  cultivate  habits  and  s kills  of 
research,  technology  integration,  and  critical  analysis.  In  addition,  according  to  Information  Power (1998).  the 
following  content  area  standards  as  applied  to  the  “Information  Literacy  Standards  for  Student  Learning”  (see  Table 
Three)  support  collaboration  of  the  library  media  specialist  with  the  health  educator. 

Content  Area  Comparisons  Between  the  School  Library  Media  Standards  and  Health  and  Related  Education 
Standards  According  to  Information  Power. 


Standard  No. 

Behavioral  Studies 

Health 

Life  Skills 

Standard  # 1 

• Understands  that 
people  can  learn 
about  others  in  many 
different  ways. 
(Grades  3-5) 

• Knows  local,  state, 
federal,  and  private 
agencies  that  protect 
and/or  inform  the 
consumer.  (Grades  9- 
12) 

• Asks  “how  do  you 
know”  in  appropriate 
situations.  (Grades 
K-2) 

Standard  # 2 

• Knows  how  to 
determine  whether 
various  sources  from 
home,  school,  and 
the  community 
present  valid  health 
information, 
products,  and 
services.  (Grades  9- 
12) 

• Knows  how  to  locate 
and  use  community 
health  information, 
products,  and 

• Analyzes  arguments 
to  determine  if  they 
are  supported  by  facts 
from  books,  articles, 
and  databases. 

(Grades  3-5) 

• Uses  tables,  charts, 
and  graphs  in 
constructing 
arguments.  (Grades 
9-12) 
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services  that  provide 
valid  health 
information.  (Grades 

6^8) 

Standard  #3  • Knows  how  to  locate  • Uses  tables,  charts, 

and  use  community  and  graphs  in 

health  information,  constructing 

products,  and  arguments.  (Grades 

services  that  provide  9-12) 

valid  health 
information.  (Grades 

6^8) 

Standard  #4  • Knows  techniques  • Compares  consumer 

for  seeking  help  and  products  on  the  basis 

support  through  of  features, 

appropriate  performance, 

resources.  (Grades  durability,  and  cost 

3-6)  and  considers 

personal  tradeoffs. 

(Grades  6-8) 

Standard  #5  • none  that  fit  directly 

with  health  and 
related  education 

standards 

Standard  #6  • Knows  a variety  of  • Reformulates  a new 

consumer  influences  hypothesis  for  study 

and  how  those  after  an  old 

influences  affect  hypothesis  has  been 

decisions  regarding  eliminated.  (Grades 

health  resources,  6-8) 

products,  and 

services  (e.g.,  media, 

information  from 

school  and  family, 

peer  pressure). 

(Grades  3-5) 

Standard  #7  • none  that  fit  directly 

with  health  and 
related  education 

standards 

Standard  #8  • none  that  fit  directly 

with  health  and 
related  education 

standards 

Standard  #9  • Knows  how  refusal,  • Adjusts  tone  and 

negotiation,  and  content  of 

collaboration  skills  information  to 

can  be  used  to  avoid  accommodate  the 

potentially  harmful  likes  of  others, 

situations.  (Grades  (Grades  K-12) 

9-12) 

(Information  Power,  p.  8 -43) 
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The  following  charges  to  the  school  library  media  specialist  illustrate  how  these  two  groups  can  work 
together  to  better  support  their  students’  needs: 


Working  With  Health  Educators:  The  School  Library  Media  Specialists’  Charge 

Collaboration 


• establish  a good  relationship  with  teachers 

• inform  teachers  of  what  the  media  center  can  do 

• Become  aware  of  health  literacy  skills 

• show  the  connections  between  information  literacy  and  the  content-related  objectives  (see  above) 

• solicit  health  teachers’  assistance  in  library  media  program  development 

• be  flexible  in  expectations  and  timing 

• work  with  the  health  educator  to  integrate  the  SLMC  (School  Library  Media  Center)  into  the  curriculum 

• provide  intellectual  and  physical  access  to  information  and  ideas 

• provide  a climate  conducive  to  learning 

• Provide  ongoing  assessment  in  its  work  with  health  education 

• Provide  funds  for  obtaining  needed  curricular  materials  and  support  for  health  education 

• Solicit  input  from  health  educators  regarding  materials’  selection,  purchase,  and  evaluation 

• Provide  ongoing  staff  development  to  health  educators 

Leadership 

• demonstrate  the  variety  of  media  center  materials  and  equipment  to  the  health  educator/  students 

• instruct  the  health  educator/students  in  how  to  access  and  use  these  materials  and  equipment 

• promote  collaborative  planning,  curriculum  development,  and  collaborative  teaching 

• support  the  diverse  learning  needs  of  all  students 

• promote  the  SMLC  program  as  an  essential  link  to  the  health  education  community 

• promote  flexible  and  equitable  access  to  information,  ideas,  and  resources  for  learning 

• support  intellectual  freedom/  intellectual  properties 

• reflect  legal  guidelines  and  professional  ethics  of  all  kinds 

• Communicate  clearly  all  goals  and  objectives  of  the  SLM  program 

Technology 


• focus  on  technology  as  a process  rather  than  as  a product 

• use  technological  processes  and  resources  to  enhance  learning 

• collaborate  with  the  health  educator  to  design  and  develop  student  experiences  that  focus  on  authentic 
learning,  information  literacy,  and  curricular  mastery 

• integrate  the  uses  of  technology  for  teaching  and  learning 

Conclusion 

Never  before  has  there  been  such  an  opportunity  and  demand  for  School  Library  Specialists  and  Health 
Teachers  to  work  together  to  promote  health  and  information  literacy  skills  in  students.  With  the  near  omnipresence 
of  computers  and  the  rapid  proliferation  of  web  sites,  students  are  more  than  ever  required  to  have  skills  to  critically 
evaluate  what  they  see,  read,  and  hear.  Cultural  norms  are  not  only  perpetuated  through  media  outlets  such  as  radio, 
television,  film,  music,  billboards,  news  and  magazine  articles  and  ads,  but  through  the  constant  intermingling  of 
advertising,  information,  and  visual  signs  and  symbols  found  on  the  internet.  The  student  has  become,  more  than 
ever,  a consumer  whose  challenge  is  to  sort  through  the  constant  bombardment  of  images  and  messages  that  have 
often  been  placed  there  under  no  professional  or  ethical  standards.  What  is  taught  in  the  health  classroom  through 
traditional  media  techniques  such  as  teacher-  prepared  overhead  transparences  or  even  the  more  visually  stimulating 
Power  Point  slides,  are  countered  during  class  breaks,  out  of  school  and,  now,  even  during  class,  with  thousands 
more  media  messages  that  are  infiltrating  students'  way  of  thinking  about  the  world  and  their  place  in  it.  The 
decreasing  cost  of  palm  computers  will  only  increase  the  opportunities  for  students  to  have  continual  access  to 
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unfiltered  messages  about  what  is  hip,  trendy,  and  cool.  These  trends— promoted  through  advertising  or  unchallenged 
websites  and  emails -affect  students'  mental,  emotional,  social  and  physical  health.  It  is  now  incumbent  upon 
students  to  have  the  critical  skills  to  sort  through  those  messages  that  are  in  their  best  interests  and  those  that  would 
exploit  them  for  economic  or  even  political  gain. 

Lest  one  question  the  relationship  between  media  messages  and  physical  health,  it  is  not  far  fetched  to 
consider  at  least  two  patterns:  the  increasing  use  by  students  of  computers  and  other  technological  devices  that  their 
time,  motivation,  and  need  to  physically  move;  and  the  proliferation  of  websites  and  unsolicited  email  messages 
about  diets  and  weight  control  that  promote  unrealistic  and  unhealthy  body  images  and  the  purchase  of  dangerous 
diet  products  and/or  techniques. 

Mentally,  socially  and  emotionally,  there  is  no  question  that  media  communications  give  students  mixed  and  often 
damaging  messages  about  what  is  appropriate  to  think,  feel  and  do.  Fortunately,  many  students  are  capable  of 
dismissing  unhealthy  or  risky  images  and  messages,  and  a lot  of  older  teens  have  the  skills  to  delineate  between 
those  that  are  presented  for  entertainment  or  shock  value  and  those  that  have  validity.  For  those  who  don't  have 
those  skills,  however,  the  challenge  for  the  1 health  educator  and  library  specialist  is  to  design  programs  -school  and 
community-based-that  actively  involve  young  people  in  critically  evaluating  the  culture,  the  media,  and  technology 
itself 

As  university  -level  preparation  specialists  in  the  two  fields,  we  recommend  that  school  health  and 
library/media/technology  specialists  collaborate  to  accomplish  shared  goals,  as  described  below. 

1.  Determine  general  information  sources  and  technological  innovations  that  impact  the  health  knowledge  and 
skills  of  children  and  adolescents.  This  will  involve  analyzing  the  many  ways  health  content-such  as  found  in 
mental  and  emotional  health,  substance  use  and  abuse,  and  consumer  health  units,  among  others  - intersect  with 
media  influences  and  technological  innovations.  For  example,  in  what  ways  should  the  study  of  illicit  drugs 
include  valid  research  data  that  can  provide  a context  for  how  it  is  that  illegal  substances  travel  from  drug- 
producing  regions  to  local  communities?  How  does  the  media  portrayal  often  suicide  impact  students'  attitudes 
and  behaviors  regarding  this  tragic  act?  What  are  the  ways  that  innovations  such  as  technological  devices, 
gadgets,  hardware  and  software  diffuse  to  diverse  population  groups?  Are  these  innovations  necessary  or  only 
desired? 

2.  Design  strategies  to  educate  students  about  media  and  information  literacy.  This  would  include  teaching 
children  and  adolescents  techniques  for  critically  analyzing  information  sources  and  media  sources  for  the 
messages,  the  connotations,  and  the  denotations.  In  addition,  this  would  involve  teaching  students  skills  in  using 
internet  applications,  such  as  evaluating  web  sites;  developing  appropriate  and  ethical  web  quests;  and 
designing  multimedia  presentations,  such  as  documentaries  or  docudramas  to  educate  others  about  cultural  and 
media  norms  that  impact  health  and  well-being. 

3.  Addressed  these  educational  strategies  in  a broad-based  fashion.  That  is,  determine  the  extent  to  which 
individuals  and  groups  or  classes  of  students  can  work  in  school  with  the  Library /Media/Technology  specialist 
in  computer  applications,  internet  and  data-base  searches,  media  criticism,  and  multimedia  development,  among 
many  other  activities.  In  addition,  involve  students  in  designing  outreach  and  advocacy  programs  that  will 
provide  students,  staff  and  administrators,  parents,  teachers,  and  community  members  with  information,  tools, 
and  skills  to  become  savvy  and  lifelong  learners/investigators,  socially  responsible  citizens,  and  informed 
consumers. 

While  the  authors  strongly  recommend  that  health  teachers  and  LMS' s work  across  disciplines  and  school 
programs  to  ensure  that  school  personnel  are  unified  in  their  approach  to  media/technological  literacy,  a discussion 
of  such  a multi-pronged  approach  is  beyond  the  scope  of  this  paper.  Suffice  it  to  say  that  the  health  and  information 
literacy  needs  of  students  can  begin  to  be  met  through  coordinated  efforts  between  the  school  or  district  LMS  and 
the  district  health  coordinators.  Current  professional  preparation  standards  indicate  that  specialists  in  each  field  need 
to  be  able  to  determine  the  health  and/or  literacy  needs  of  adolescent  populations;  design  appropriate,  ethical,  and 
standards -based  programs  that  address  these  needs;  implement  such  programs  in  collaboration  with  other  academic 
and  community  personnel/  agencies;  and  continually  evaluate  the  effectiveness  of  such  programs.  It  is  recommended 
and  it  is  feasible  for  professionally  prepared  school  health  educators  and  school  LMS's  to  work  together  to 
strengthen  students'  knowledge  of  and  ability  to  use  media  and  technology  appropriately.  This  can  and  will  have  an 
immense  and  positive  impact  on  the  intellectual,  social,  emotional,  and  physical  health  of  our  children  today. 


76 


References 

Information  Power:  Building  Partnerships  for  Learning.  (1998).  Chicago:  American  Library  Association. 
Kolbe’s  Coordinated  School  Health  Model. 

National  Health  Education  Standards. 


77 


The  Effectiveness  of  Simulation  in  an  On-line  Networking  Course 
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Abstract 

Simulations  are  an  important  part  of  many  traditional  classes  in  Computer  Networking.  As  web-based 
learning  environments  grow  in  popularity  the  need  for  simulations  has  become  pronounced.  This  paper  reports  on 
a study  to  compare  the  performance  of  students  enrolled  in  two  web-based  learning  environments,  one  with  a 
simulation  package  and  the  second  with  graphics  and  text  only.  Analysis  shows  statistically  significant 
improvements  in  performance  in  the  simulation  group  compared  to  the  other  group. 


Introduction 

The  Computer  networking,  as  defined  as  the  interconnection  of  computers  and  computing  equipment  using 
either  wires  or  radio  waves  over  small  or  large  geographic  areas  (White,  2002),  has  long  been  regarded  as  one  of  the 
more  difficult  technology-related  subjects  to  teach.  Historically,  this  type  of  course  was  thought  to  require  much 
hands-on  interaction  with  the  instructor  and  was  not  viewed  as  a good  candidate  for  an  online  course.  Due  to  recent 
advances  in  network  simulation  technologies,  this  view  is  being  challenged.  New  simulations  make  complex 
networking  systems  easily  modeled  on  desktop  computers.  The  effectiveness  of  simulations  and  the  challenges  of 
distance  learning  environments  are  described  next  followed  by  the  research  design,  methods,  and  results. 

Effectiveness  of  Simulations 

Simulations  have  been  an  effective  tool  in  learning  environments  (Lindstrom  et  al.  (1993)  Williamson  & 
Abraham  (1995).  Simulation  research  can  be  divided  into  three  subsets:  cognitive  studies,  attitudinal  studies,  and 
retention  studies  (Dekkers  & Donnatti,  1981).  Cognitive  studies  are  concerned  with  the  simulation’s  effect  on  the 
cognitive  development  of  the  student.  Attitudinal  simulation  studies  are  concerned  with  student  attitude  formation 
and  retention  studies  are  concerned  with  information  retention.  This  project  touched  on  all  three  of  these  areas  of 
simulation  research.  Comparisons  were  made  of  the  cognitive  development  of  the  two  course  sections  through 
different  networking  labs  and  team  problems.  Attitudinal  development  was  accessed  through  a course  survey  given 
at  the  conclusion  of  the  course.  Retention  was  accessed  through  tests  given  at  specified  points  in  the  course. 

While  computers  afford  the  design  of  highly  interactive  open-ended  learning  environments,  such  as 
simulations,  decisions  about  how  to  design  the  instruction  used  in  conjunction  with  the  simulation  are  often  made 
with  little  understanding  of  how  the  user  will  perceive,  process  and  interpret  the  resulting  feedback  that  the 
simulation  provides.  Tailoring  the  simulation  to  the  needs  and  goals  of  a particular  course  requires  an  understanding 
of  the  capabilities  of  the  simulation  and  the  ways  it  introduces  and  enforces  various  networking  concepts.  Research 
demonstrates  that  the  way  information  is  represented  matters  greatly  in  the  learning  process  (Rieber,  1996). 

However,  these  studies  on  the  effectiveness  of  simulations  in  learning  environments  compared  simulations  with 
traditional  classroom  instruction.  Web-based  distance  learning  courses  offer  a unique  challenge  to  teaching 
traditionally  hands-on  courses  like  network  simulations. 

The  Challenge  of  Weh-Based  Distance  Learning  Courses 

Lack  of  motivation  and  low  student  satisfaction  have  been  cited  as  factors  relating  to  the  high  drop-out  rate 
in  online  education.  There  are  a number  of  studies  indicating  that  simulations  can  create  considerable  motivation  in 
their  participants  (Drawer,  1982).  Until  recently  only  simple  simulations,  with  a low  level  of  complexity,  were 
possible  in  an  online  learning  environment.  The  ability  of  this  new  online  delivery  mode  for  simulation  technology 
has  great  promise  for  many  types  of  online  courses.  A key  factor  cited  in  improving  student  motivation  when  using 
simulations  in  instruction  is  the  relevancy  or  realism  of  the  simulated  environment  (Orbach,  1979).  Advances  in 
commercially  available  network  simulations  have  produced  products  that  emulate  real-world  network  operations. 
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Research  Design 

The  purpose  of  this  research  was  to  compare  students'  performance  on  simulation  -based  courses  and  static 
graphic  representational  teaching  of  the  same  course  content  in  an  online  learning  environment. 

Participants  & Methods 

85  freshman  and  sophomores  enrolled  in  two  sections  of  a new  online  introductory  networking  and 
telecommunications  course  were  compared.  One  section  utilized  a network  simulation  package,  created  by 
NetCracker  Technologies,  Inc.,  that  allows  the  student  to  build  and  send  data  through  different  network 
configurations.  The  second  section  used  Microsoft  Visio,  a static  network  diagramming  software  package.  The  two 
groups  were  compared  and  assessed  on  a variety  of  levels. 

NetCracker  Technologies  offers  a simulation  package  that  allows  students  to  design  local  area  networks 
(LANs),  metropolitan  area  networks  (MANs),  and  wide  area  networks  (WANs)  utilizing  a wide  variety  of 
networking  components.  A wide  variety  of  data  types  may  be  used  to  test  the  functionality  of  the  network  design. 

Static  network  diagramming  software  packages  like  Microsoft  Visio  have  been  used  in  networking  courses 
for  decades  to  teach  networking  concepts  and  develop  visual  representations  of  networks. 

Description  of  the  Project 

An  undergraduate  level  course  on  Introduction  to  Networking  and  Telecommunications,  is  taught  at  a 
national  university.  The  course  comprised  of  online  instructional  materials  and  assignments.  Most  interactions  with 
the  instructor  were  online.  The  only  face-to-face  instructor  interactions  were  in  the  form  of  team  project  meetings 
and  team  presentations. 

One  course  section  consisting  of  forty-four  students  was  asked  to  use  the  network  simulation  package  for 
all  course  assignments  and  the  second  course  section  consisting  of  forty  students  was  asked  to  use  Microsoft  Visio 
for  all  course  assignments . Basic  instruction  was  given  on  both  packages. 

Assessment  of  learning  outcomes  was  conducted  using  multiple  choice  tests,  project  results,  and  a survey 
of  students.  The  students  completed  four  individual  networking  labs  and  two  team  problems  using  their  respective 
software  packages.  The  labs  required  students  to  demonstrate  knowledge  of  networking  rules  and  concepts  by 
devising  networking  solutions  to  given  scenarios.  The  team  projects  required  students  to  synthesize  factual, 
procedural,  and  conceptual  knowledge  about  computer  networks  in  order  to  solve  a complex  problem.  Mid-term 
and  final  exams,  consisting  of  fifty  multiple -choice  questions,  were  given  to  both  course  sections.  The  exams 
focused  primarily  on  factual  knowledge.  A qualitative  course  survey  was  administered  to  both  course  sections  at  the 
end  of  the  course. 

Results 

Standard  statistical  analysis  procedures  were  used  to  compare  differences  among  the  two  conditions. 

Results  from  ANOVA  showed  the  network  simulation  condition  outperformed  their  counterparts  in  the  static 
simulation  condition.  The  mean  on  the  test  performance  was  significantly  (p  < .05)  lower  for  the  static  simulation 
condition. 

Table  1 shows  the  descriptive  statistics  for  each  measure  used  in  the  analysis.  The  course  section  that 
utilized  that  network  simulation  software  demonstrated,  better  understanding  of  networking  concepts  and  better 
retention  of  the  course  information  than  did  the  course  section  the  did  not  use  the  simulation.  Scores  on  the 
individual  lab  assignments  were  all  significantly  (p  = .000)  higher  for  students  who  were  in  the  network  simulation 
group.  Team  projects  scores  were  significantly  (p=.001)  higher  for  the  simulation  group.  Mid-term  and  final  exam 


Group  Statistics 


condition 

N 

Mean 

Std. 

Deviation 

Std.  Error 
Mean 

EXAM  2 

simulation 

48 

96.7292 

5.5033 

.7943 

visio 

37 

84.3243 

30.9678 

5.0911 

EXAM  1 

simulation 

48 

92.2917 

7.9679 

1.1501 

visio 

37 

82.1622 

27.5051 

4.5218 

LAB  1 

simulation 

48 

26.3750 

3.0848 

.4453 

visio 

37 

18.2432 

9.5435 

1.5689 

LAB  2 

simulation 

48 

26.7917 

2.6654 

.3847 

visio 

37 

16.9730 

1 1 .2262 

1 .8456 

LAB  3 

simulation 

48 

25.5417 

3.6899 

.5326 

visio 

37 

16.5676 

10.2998 

1.6933 

LAB  4 

simulation 

48 

28.3542 

2.0158 

.2910 

visio 

37 

18.4865 

10.7590 

1.7688 

Project  2 

simulation 

48 

93.5833 

2.1020 

.3034 

visio 

37 

80.8108 

24.7115 

4.0625 

Project  1 

simulation 

48 

90.8125 

3.2463 

.4686 

scores  were  also  significantly  (p  = .008  for  final  and  p = .017  for  mid-term)  higher  for  the  students  that  used  the 
simulation. 

No  students  in  the  course  section  that  used  the  simulation  withdrew  from  the  course  due  to  failing  grades 
while  three  students  in  the  course  section  that  did  not  use  the  simulation  withdrew  from  the  course  with  failing 
grades. 

Students  that  used  the  simulation  reported  that  they  spent,  more  time  on  course  assignments  than  did  the 
students  that  used  Microsoft  Visio  (3.5  hours  vs.  2 hours)  and  reported  that  they  spent  more  time  on  assignments 
primarily  because  the  simulation  allowed  them  to  experiment  with  different  network  configurations  and  verify  the 
functionality  of  their  network  designs.  Many  students  reported  that  they  felt  that  the  simulation  helped  them  to 
understand  complex  networking  concepts  because  they  could  actually  build  and  test  the  different  components  of  a 
network  and  try  different  network  configurations.  Many  of  the  students  that  used  Microsoft  Visio  reported  that  they 
had  no  way  to  verify  that  their  network  designs  functioned  correctly  prior  to  submitting  them  to  the  instructor  and, 
as  a result,  did  not  experiment  with  different  network  configurations. 

Implications  for  Learning  Networking 

Online  learning  environments  have  long  been  plagued  with  student  motivational  problems  that  have  lead  to 
historically  high  drop-out  rates.  This  project  suggests  that  the  use  of  interactive  learning  tools,  such  as  simulations, 
have  the  potential  to  increase  student  motivation  and  learning  in  an  online  learning  environment.  Simulations  offer 
immediate  feedback  and  allow  the  learner  to  explore  different  alternatives  at  will.  Simulations  enable  knowledge 
application  through  multidimensional  problem  solving  (de  Mesquita,  1992).  Simulations  have  been  found  to 
significantly  improve  knowledge  transfer  (Kozma,  1992,  White,  1994).  This  type  of  discovery -based  learning  using 
simulations  has  been  shown  to  increase  understanding  of  abstract  concepts  (Rieber,  1996)  and  increase  student 
motivation  (Drawer,  1982). 

The  network  simulation  utilized  in  this  study  can  be  used  for  a variety  of  learning  objectives  targeted  at 
differing  levels  of  learning.  The  intellectual  skills  that  contribute  to  challenging  classroom  learning,  as  identified  by 
Gagne  (1977,  1985),  are  discrimination,  concept  learning,  and  rule  utilization.  Simulation  affords  the  instructor  the 
opportunity  to  develop  instructional  events  for  all  of  these  levels  of  learning. 

For  example,  at  the  factual  level,  the  simulation  can  be  used  to  identify  the  parts  of  a networking  system. 
The  package  can  also  be  utilized  as  a type  of  procedural  simulation,  described  by  Riegeluth  and  Schwartz  (1989), 
and  teach  network  diagramming  and  documentation  procedures.  At  the  conceptual  and  rule  levels,  the  deep 
structures  made  possible  by  the  diagnostic  capabilities  of  the  simulation  allow  for  evolving  problems  that  require 
sequential  and  interrelated  decisions.  Therefore,  as  in  real  situations,  errors  may  be  compounded  on  top  of  errors  as 
nonproductive  diagnostic  and  solution  procedures  are  pursued  (Berven  & Scorfield,  1980). 

Suggestions  for  Further  Inquiry 

While  the  use  of  simulation  in  education  has  been  studied  for  decades,  its  use  in  an  online  learning 
environment  has  not  been  widely  explored.  Most  simulation  research  compares  simulations  to  regular  classroom 
instruction.  The  instructional  goals  for  which  each  can  be  most  effective  differ  and  more  research  needs  to  be 
complied  on  the  use  of  simulation  as  a supplement  to  traditional  and  online  instruction.  Traditional  face-to-face 
instruction  and  online  instruction  can  be  effective  at  transmitting  items  of  information  while  simulations  have  the 
potential  to  develop  students’  mental  models  of  complex  systems  and  problem  solving  strategies.  Further  research 
on  the  effectiveness  of  differing  combinations  of  these  instructional  methods  is  needed. 

Another  area  of  simulation  research  that  requires  additional  study  is  the  use  of  feedback.  Schimmel  (1983)  has 
identified  three  types  of  feedback;  confirmation,  corrective,  and  explanatory.  The  simulation  utilized  in  this  study 
provided  only  confirmation  feedback.  Research  on  the  information  content  of  feedback  has  been  inconsistent  and 
has  provided  little  guidance  for  designing  feedback.  More  research  is  needed  on  the  effectiveness  of  different  types 
of  feedback  in  simulation  and  in  online  learning  environments. 

Conclusion 

The  results  reported  here  suggest  that  web-based  courses  on  computer  networking  can  utilize  simulations 
very  effectively. 


References 

Drawer,  M.P.  (1982).  Intergrating  motivational  activities  into  instruction:  A developmental  model.  16 
pages.  (ERIC  Document  Reproduction  Service  No.  ED  222  106). 


80 


Berven,  N.L.  & Scofield,  M.E.  (1980).  Evaluation  of  clinical  problem-solving  skills  through  standardized 
case-management  simulations.  Journal  of  Counseling  Psychology  27(2),  199-208. 

Dekkers,  J.  & Donatti,  S.  (1981).  The  integration  of  research  studies  on  the  use  of  simulation  as  an 
instructional  strategy.  Journal  of  Educational  Research  74,  424-27 

DeMesquits,  P.  (1992).  Diagnostic  problem-solving:  method  or  guesswork?  Journal  of  School 
Psychology  50,  269-91. 

Gagne,  R.  (1977).  The  conditions  of  learning,  3d  ed.  New  York:  Holt.  (1985).  The  conditions  of  learning, 
4**'  ed.  New  York:Holt. 

Kozma,  R.  B.  (1992).  Learning  with  media.  Review  of  Educational  Research  61(2),  179-211. 

Orbach,  E.  (1979).  Simulation  games  and  motivation  for  learning:  A theoretical  framework.  Simulation  & 
Games.  10,  3-40. 

Reiber,  L.P.  (1996).  Animation  as  feedback  in  a computer-based  simulation:  Representation  matters. 
Educational  Technology  Research  & Development,  44(1),  5-22 

Riegeluth,  C.M.  & Schwartz,  E.  (1989).  An  instructional  theory  for  the  design  of  computer-based 
simulations.  Journal  of  Computer-Based  Simulations  16(1),  1-10. 

Schimmel,  B.  (1983).  Ability  versus  effort  attributional  feedback:  Differential  effects  on  self-efficacy  and 
achievement.  Journal  of  Educational  Psychology,  75,  848-856. 

White,  B.  Y.  (1994).  A microworld -based  approach  to  science  education.  In  E.  Scanlon  & T.  O’Shea,  eds. 
New  directions  in  educational  technology.  New  York:  Springer. 

Joyce,  D.  SIGCSE  ’99  format  instructions,  1998.  Online.  Internet.  [June  15,  1999].  Available  WWW: 
http://www.cs.wm.edu/sigcse99/format.doc 


81 


Students’  Learning  Styles  and  Collaboration  in  Project-based  Learning  of 

Web  Page  Development 

Li- Ling  Chen 


Abstract 

Project-based  learning  theorists  advocate  that  learning  is  promoted  when  students  pursue 
individual  interests,  when  they  build  on  prior  knowledge,  and  when  they  engage  in  hands-on  and  authentic 
activity.  Although  a great  deal  of  literature  exists  describing  ideals  such  as  these,  research  examining  the 
implementation  of  these  ideals  in  classrooms  is  scarce.  The  purpose  of  the  study  is  to  investigate  the  effects 
of  collaboration  and  students’  learning  style  on  field  dependency  while  involving  in  project-based  learning 
activities  in  designing  and  developing  Web  pages.  A group  of  K-12  teachers  who  registered  a graduate 
course  on  Web  page  development  were  asked  voluntarily  to  serve  as  participants.  Both  quantitative 
{learning  style  test  and  Web  page  projects  analysis)  and  qualitative  data  (teacher’s  log  of  the  activities, 
students’  emails,  and  interview)  were  collected. 


Introduction 

During  the  past  few  years,  teaching  learners  to  develop  Web  Pages  has  proliferated  in  both  education  and 
business.  There  are  tremendous  numbers  of  interdisciplinary  learners  taking  Web  page  development  classes.  The 
learners  include  K-12  students,  college/university  students,  or  any  adults  who  are  interested  in  the  World  Wide  Web 
(WWW).  Although  there  is  a huge  difference  among  individual  learners,  there  is  one  common  instructional  strategy 
that  has  been  shared  by  all  of  the  instructors.  That  strategy  is  project -based  learning.  In  other  words,  almost  all 
learners  who  teach  Web  authoring  classes  are  required  or  recommended  to  design  and  develop  Web  pages. 

Project-based  learning  is  an  instructional  strategy  in  which  students  generate  projects  as  part  of  their 
learning  process.  Students  assume  the  role  of  active  learners  as  they  construct  their  own  knowledge  and  plan  and 
develop  their  projects.  Project-based  learning  is  supported  by  the  constructivist  view  of  learning.  Students  are 
provided  with  real  world  contexts  and  problem-solving  situation  to  make  learning  experience  meaningful. 

Project-based  learning  theorists  advocate  that  learning  is  promoted  when  students  pursue  individual 
interests,  when  they  build  on  prior  knowledge,  and  when  they  engage  in  hands-on  and  authentic  activity.  Although  a 
great  deal  of  literature  exists  describing  ideals  such  as  these,  research  examining  the  implementation  of  these  ideals 
in  classrooms  is  scarce. 

“The  information  technologies  increase  the  versatility  and  value  of  project-based  learning  as  a curriculum 
tool.  Technology  can  help  create  a rich  environment  for  individuals  and  teams  to  carry  out  in-depth  projects  that 
draw  on  multimedia  and  information  resources  from  throughout  the  world’’  (NFIE,  2002). 

Learning  Web  authoring  via  a project-based  approach,  especially  with  collaboration  method,  provides 
learners  a social  interaction  opportunity  to  organize  their  thoughts  and  create  their  own  meaningful  learning 
experiences.  Since  field  dependent  students  are  “socially  sensitive  and  interpersonal  in  orientation’’  (Witkin  & 
Goodenough,  1979),  will  they  learn  better  with  such  instructional  approach?  It  is  also  interesting  to  study  the  impact 
of  such  instructional  strategies  on  field  independent  learners.  It  is  based  on  the  arguments  that  the  researcher 
proposes  studying  the  influence  and  interaction  between  learners’  field  dependence/  independence  and  collaboration 
on  their  performance  in  developing  Web  pages. 


Purpose  of  the  Study 

The  purpose  of  the  study  is  to  investigate  the  effects  of  collaboration,  individualization,  and  students’ 
learning  style  in  field  dependency  while  involving  in  project-based  learning  activities  in  designing  and  developing 
Web  pages.  Specific  questions  include: 

1.  What  influences  do  collaboration  and  individualization  lave  on  students’  learning  of  Web  page 
development  in  a project-based  learning  environment? 

2.  How  does  students’  learning  style,  specifically  field  dependence/independence,  affect  their  learning  of 
Web  page  development  in  a project-based  learning  environment? 

3.  What  is  the  relationship  between  instmctional  strategies  of  collaboration  and  individualization  and 
learners’  learning  styles  in  field -dependency  and  field -independency  in  a project-based  learning 
environment  on  Web  page  development? 
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Project- based  Learning 


Definition  and  Theory  - According  to  the  project-based  learning  (PBL)  handbook  published  by  Bulk  Institution  of 
Education  (2002),  the  defining  features  of  project-based  learning  include  content,  conditions,  activities,  and  results. 
First,  contents  focus  on  compelling  ideas.  “Project  Based  Learning  is  worthwhile  because  it  allows  teachers  and 
students  to  focus  in  depth  on  central  ideas  and  salient  issues.”  With  project-based  learning,  problems  presented  in 
their  full  complexity;  content  can  be  presented  realistically,  holistically,  rather  than  in  fragments,  and  investigated  in 
depth. 

Second,  “conditions  encourage  social,  personal  and  collaborative  skills.  Project  Based  Learning  can  give 
students  a richer,  more  ''authentic''  learning  experience  than  other  learning  modes  because  it  occurs  in  a social 
context  where  interdependence  and  cooperation  are  crucial  for  getting  things  done.  This  context  also  allows 
students  to  prevent  and  resolve  interpersonal  conflicts.  In  a non-threatening,  supportive  environment,  students  gain 
the  confidence  to  develop  their  individual  abilities.”  Conditions  also  “encourage  mastery  of  technological  tools. 
Projects  provide  an  ideal  context  for  learning  to  use  computer  technology  and  graphic  arts  tools,  thus  extending 
students'  capabilities  and  preparing  them  for  the  world  beyond  school.  Using  technology:  expands  students' 
capabilities  to  display  and  manipulate  information  widens  students'  interests  and  career  options  multiplies  the  ways 
that  individual  students  can  contribute  to  project  work.” 

Third,  activities  are  effective,  engaging  strategies.  In  project-based  learning,  “students  tackle  difficult 
questions  or  problems.  Investigations  provide  students  opportunities  to:  learn  complex  ideas  and  skills  in  realistic 
settings  apply  skills  to  a variety  of  contexts  combine  skills,  by  completing  "expert"  tasks,  professional  duties,  job 
performance,  or  reaHife  demonstrations  solve  problems.” 

Fourth,  results  mean  real  world  outcomes.  For  example,  students  generate  complex  intellectual  products 
that  demonstrate  their  learning  (e.g.,  models,  reports);  students  participating  in  their  own  assessment;  students  held 
accountable  for  choosing  how  they  will  demonstrate  their  competence;  and  students  exhibit  growth  in  frequently 
neglected  areas  important  for  real-world  competence:  social  skills,  life  skills,  self- management  skills,  and 
dispositions  to  learn  on  one's  own.  “Results  also  include  certain  skills,  dispositions,  attitudes,  and  beliefs  associated 
with  productive  work.  Project  Based  learning  can  effectively  accomplish  goals  that  are  difficult  to  achieve  with 
other  models  of  instruction.” 

PBL  and  Internet 

A great  deal  of  literature  (Bartscher,  et  al.,  1995;  Cognitive  and  Technology  Group  at  Vanderbilt,  1992; 
Gonella,  2000;  Land  & Greene,  2000;  Nicaise  & Crane,  1999;  Meyer,  et  al.,  1997)  has  suggested  that  project-based 
learning  with  hypermedia  technology  such  as  WWW  has  great  potential  to  enhance  students’  motivation  and 
learning.  Its  benefits  include:  motivating  students,  increasing  students’  mental  efforts,  involvement,  interest, 
planning,  and  developing  students’  higher  order  thinking  skills.  On  the  other  hand,  there  are  also  many  researchers 
concluded  that  the  instructional  effect  of  project-based  learning  with  the  WWW  is  questionable  (Land  & Greene, 
1999;  Wang,  et  al.,  2000;  Laffey,  et  al.,  1998;  Ward,  1997). 

In  addition,  project-based  learning  can  be  conducted  collaboratively  or  individually.  It  is  interesting  to  find 
that  research  examining  the  impact  of  collaboration  versus  individualization  in  a project-based  learning  environment 
on  Web  authoring  is  scarce,  and  research  investigating  the  relationship  between  learners’  field  dependence  and 
collaboration  is  even  scarcer. 

Research  (Bulk  Institute  of  Education,  2002)  also  concluded  that  project-based  learning  helps  students  gain 
self-confidence  from  their  project  work,  become  motivated,  self-directed  learners  in  other  contexts.  “Additional 
studies  of  PBL  effectiveness  have  emerged  from  the  staff  of  the  Cognitive  and  Technology  Group  at  Vanderbilt 
University  (CTGV,  1992).  These  studies  involve  video-based  stories  that  introduce  complex  problems  or  project 
ideas  to  students.  Although  these  studies  have  demonstrated  gains  in  students'  skills  and  understanding  of  project 
content  as  a result  of  PBL  experience,  their  greatest  value  may  be  in  establishing  principles  and  methods  to  improve 
the  process  of  PBL.” 

PBL  and  Students’  Learning  Styles  -Researchers  studied  the  role  of  student  characteristics  in  project-based 
learning  believes  that  there  is  a differential  appropriateness  or  effectiveness  of  PBL  for  different  kinds  of  students. 
Several  PBL  practitioners  concluded  that  “PBL,  because  of  its  various  features,  is  a more  effective  means  of 
adapting  to  students'  various  learning  styles  or  "multiple  intelligences"  (Gardner,  1991)  than  is  the  traditional 
instructional  model  (Diehl,  et  al.,  1999).” 
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A review  of  literature  concluded  that  only  four  studies  were  found  that  investigated  the  role  of  individual 
differences  in  Project-Based  Learning.  Rosenfeld  and  Rosenfeld  (1998)  investigated  the  learning  styles  of  students 
who  were  characterized  by  their  teachers  as  "pleasant  surprises"  (students  who  perform  poorly  in  conventional 
classrooms,  but  who  do  well  in  PBL  activities)  and  "disappointing  surprises"  (students  who  performed  well  in 
conventional  classrooms,  but  who  turned  in  poor  projects  or  no  projects  at  all).”  Their  study  concluded  that 
"pleasant  surprises"  students  have  a better  academic  achievement  for  applied,  discovery,  technical,  and  confluent 
processing  in  a math  lesson  test,  whereas  students  who  were  characterized  as  "disappointing  surprises"  scored  high 
on  the  fact-oriented  information. 

A second  study,  conducted  by  Meyer,  Turner,  and  Spencer  (1997)  investigated  the  relationship  between 
students’  learning  style  and  PBL.  They  divided  a group  of  fifth-  and  sixth-grade  students  into  "challenge  seekers" 
versus  "challenge  avoiders"  based  on  surveys  and  interviews,  and  found  that  "challenge  seekers  would  approach 
project-based  learning  with  greater  interest  and  mastery  focus  than  would  "challenge  avoiders." 

The  third  study,  conducted  by  Horan,  Lavaroni,  and  Beldon  (1996),  compared  the  behavior  of  high  ability  to  low 
ability  PBL  students  in  group  problem-solving  activities,  and  concluded  that  “high-ability  students  engaged  in  the 
criterion  social  participation  behaviors  more  than  two  and  one-half  times  as  frequently  as  low-ability  students  in  the 
four  classes  observed  and  engaged  in  critical  thinking  behaviors  almost  50%  more  frequently.  The  interesting 
finding,  however,  was  that  lower  ability  students  demonstrated  the  greatest  gain  in  critical  thinking  and  social 
participation  behaviors,  an  increase  of  446%  between  two  different  observations,  compared  to  an  increase  of  76% 
for  the  high-ability  students.” 

The  fourth  study,  by  Boaler  (1997),  investigated  mathematics  learning  in  two  contrasting  schools  and  found 
differences  between  girls  and  boys  in  their  preferred  mode  of  learning.  They  found  that  girls  were  “more  disaffected 
by  traditional  instruction  than  boys  and  showed  lower  achievement  than  a matched  sample  of  girls  taught  with 
project-based  methods.” 

The  literature  review  revealed  that  none  of  the  research  has  studied  students’  cognitive  learning  styles  on  field 
dependency  and  field  independency  in  a project-based  learning  approach.  This  is  one  of  the  reasons  to  support  the 
author’s  study  on  the  issue. 

Use  of  Cooperative  and  Individualized  Learning  Strategies  for  Project-based  Learning 

Project-based  learning  can  be  carried  out  individually  or  cooperatively.  In  other  words,  teachers  can  have 
their  students  work  on  a project  individually  or  in  small  groups.  Literature  suggests  that  projects  organized  by 
cooperative  groups  can  increase  students’  motivation  (Bartscher,  1995).  There  are  lots  of  studies  on  the  use  of 
collaborative  learning  strategy  for  project-based  learning  (Peterson  & Myer,  1995);  however,  none  of  the  study  was 
found  to  investigate  the  effect  of  students’  characteristics  and  cooperative  learning  strategies  in  a project-based 
learning  environment. 

Field-independent  and  Field-dependent  Learners 

The  impact  of  learners’  cognitive  styles,  defined  as  “characteristics  modes  of  perceiving,  remembering, 
thinking,  problem  solving,  decision  making”  (Messick,  1993,  p.3),  in  teaching  and  learning  has  long  gained  its 
attention.  According  to  Riding  and  Cheema  (1991,  p.  195),  a cognitive  style  is  a "fixed  characteristic"  of  an 
individual  that  is  developmental,  static,  and  stable. 

Learners’  characteristics  and  tendencies  have  been  classified  into  numerous  constructs  in  the  literature  as 
part  of  the  cognitive  style.  Examples  of  the  cognitive  styles  include  auditory  learners,  visual  learners,  kinesthetic 
learners,  and  so  on.  Field  independent/dependent  construct,  began  in  the  1940s  with  Herman  Witkin’s  research  on 
human  perception  of  the  upright  (Witkin  & Goodenough,  1979;  Witkin,  Moore,  Goodenough,  & Cox,  1977)  is  one 
of  the  classification. 

Individuals  who  were  able  to  orient  themselves  along  the  true  vertical  in  a room  despite  confusing  physical 
and  visual  cues  generated  by  a tilted  floor  and  moveable  chair  were  described  as  articulated  or  field  independent. 
Witkin  (1979,  1977)  defined  field  independent  learners  are  “individuals  who  were  able  to  orient  themselves  along 
the  true  vertical  in  a room  despite  confusing  physical  and  visual  cues  generated  by  a tilted  floor  and  moveable  chair 
were  described  as  articulated.”  Field  dependent  learners  are  “individuals  who  aligned  themselves  along  a vertical 
axis  relative  to  the  misleading  environment.”  Witkin  and  Goodenough,  in  1981,  further  indicated  that  the  field- 
independent  learners  are  “analytic,  self-referent,  impersonal  in  orientation”,  whereas  the  field  -dependent  learners  are 
“global,  socially  sensitive,  and  interpersonal  in  orientation.” 

“Field  dependence/independence  is  generally  considered  to  describe  learners  along  a testable,  value  neutral, 
bipolar  continuum  such  that  individuals  at  one  end  are  measured  as  field  independent.  Individuals  at  the  opposite 
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end  are  considered  field  dependent,  and  subjects  in  the  middle  of  the  range  are  characterized  as  field  mixed  or  field 
neutral  (Hall,  2000).” 

Research  indicated  that  “field  dependent  learners  generally  perform  less  well  than  field  independent 
individuals  in  most  instructional  environments”  and  are  generally  disadvantaged  (Hall,  2000).  Field  independent 
learners  are  also  more  efficient  information  processors  with  better  short  term  memory  encoding,  better  long  term 
recall,  and  more  accurate  performance  on  visual  search  tasks  than  field  dependent  individuals  (Davis,  1991). 

There  are  numerous  studies  exploring  the  importance  of  learners’  cognitive  styles  and  the  role  of  field 
dependence/independence  in  teaching  and  learning  across  academic  disciplines  and  at  all  levels  of  schooling 
(Burton,  Moore,  & Holmes,  1995;  Hall,  2000).  However,  the  consequences  of  cognitive  style  differences  have  not 
been  thoroughly  pursued  by  educators  in  project-based  learning  environment  for  Web  authoring. 

Educational  theorists  have  demonstrated  that  some  instructional  strategies  are  more  or  less  effective  for 
particular  individuals  depending  upon  their  specific  characteristics.  Dyer  and  Osborne’s  (1996)  study  offers  another 
example.  They  concluded  that  field  independent  learners  learn  significantly  better  with  a problem-solving  approach 
than  a subject-matter  approach. 

It  is  suggested  that  technology  may  impact  learners  more  in  some  situations  than  it  does  in  others  because 
of  the  interaction  between  the  task,  the  learners,  and  the  technology  (Kozma , 1991).  This  view  is  compatible  with 
Cronbach  and  Snow’s  (1977)  argument  that  for  some  instructional  strategies  are  more  or  less  effective  for  particular 
individuals  depending  upon  their  specific  characteristics. 

A review  of  literature  suggested  that  field -dependent  learners  were  slightly  more  satisfied  with  ICN 
((Interactive  Communications  Network)  instruction  overall,  but  there  was  not  significant  difference  between  the 
attitude  scores  of  field-dependent  and  filed- independent  learners  toward  distance  learning  (Miller,  1997).  Although 
many  researchers  (Anderson,  2000;  Burton,  Moore,  & Holmes,  1995;  Jonassen  & Wang,  1993;  Luk  Suet  Ching, 
1998;  Weller,  Repman,  Lan,  & Rooze,  1995;)  argue  that  field  independent  students  are  more  likely  to  benefit  from 
Web-based  learning  experiences  than  field  dependent  students,  their  researches  focus  on  students’  learning  by 
viewing  hypermedia  programs,  instead  of  authoring. 

Web  page  developing  is  a learning  and  designing  process  to  display  associative,  possibly  nonlinear 
information  built  around  a network  of  multimedia  materials.  It  heavily  depends  on  a learner’s  ability  to  determine 
and  control  selection  and  non-linear  sequence  of  content;  it  is  composed  of  user-determined  (associative)  links 
which  add  up  to  an  individual  navigational  trail. 

Literature  indicated  that  “field  independents  are  more  likely  to  have  an  internal  locus  of  control  than  field 
dependents,  but  field  dependents  are  more  likely  to  be  successful  self-monitors  in  a social  group”  (Leventhal  & 
Sisco,  1996).  In  educational  situations,  field  dependent  learners’  tendency  to  be  influenced  by  their  peers  is  critical 
as  they  prefer  feedback  and  social  sources  of  information  (Jones,  1993).  Field  independent  learners  are  more 
individualistic  and  mle -oriented  and  less  likely  to  seek  peer  input  (Jones,  1993).  Does  the  literature  imply  that  field 
dependent  learners  will  learn  better  in  a cooperative  learning  environment  than  field  independent  learners? 


Methods  and  Procedures 

This  study  examined  the  effect  of  collaboration  and  students’  learning  style  in  a project-based  learning 
environment  on  Web  page  development.  Two  graduate  courses  on  Web  page  development  was  implemented  using 
a project-based  approach  and  offered  in  the  Summer  Quarter  of  2002  at  the  California  State  University  at  Hayward. 
Both  of  the  classes  were  taught  by  the  same  instructor,  and  the  instructor  introduced  the  same  hypermedia  design 
principles  and  Web  authoring  tool,  Macromedia  Dreamweaver.  As  to  Web  projects,  students  were  required  to 
include  at  least  a navigation  mechanism  by  which  learners  control  the  order  and  flow  of  information,  multiple 
formats  of  presenting  their  materials,  and  some  types  of  interactivity. 

Participants  were  the  students  who  enrolled  in  the  courses  and  also  taught  in  K- 12  educational 
environments.  There  were  twenty-eight  voluntary  participants.  Participants  were  allowed  to  carry  out  Web  projects 
individually  or  collaboratively.  If  choosing  to  do  the  Web  projects  collaboratively,  only  two  members  in  each  group 
were  allowed.  Attitudinal  and  demographic  data  were  collected  through  face-to-face  interviews  or  email  interviews. 

Both  quantitative  and  qualitative  data  were  collected.  Quantitative  data  include  students’  field  dependency 
from  learning  style  test  and  their  Web  project  scores  from  analyzing  their  Web  projects.  Specifically,  learners’ 
preferred  cognitive  styles  on  field  dependency  or  field  independency  were  measured  with  Group  Embedded  Figure 
Test  (GEFT),  developed  by  Consulting  Psychologists  Press,  Inc.,  which  is  a testing  instrument  to  identify  whether  a 
learner  is  field  dependent  or  field  independent.  Students’  Web  projects  were  evaluated  with  a multimedia  project 
scoring  rubrics,  designed  and  developed  by  the  author  to  meet  its  special  needs  (See  attachment).  In  the  rubric,  the 
instructor  allocated  a score  for  each  category.  Therefore,  a final  score  will  be  attained  for  each  Web  project. 
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Qualitative  data  was  also  collected  in  order  to  increase  the  reliability  of  the  quantified  Web  project  data. 
The  researchers  collected  the  instructor’s  observation  and  log  of  students’  activities,  students’  email  conversation, 
and  interviewing  with  students  on  such  learning  experiences. 

T test  was  used  to  compare  field  dependent  learners’  performance  in  developing  Web  pages  with  field 
independent  learners,  and  also  used  compare  learners’  performance  between  collaboration  and  individual  strategies. 
Collected  qualitative  data  was  chucked  and  analyzed  to  interpret  the  impact  and  interaction  of  learners’  learning 
styles  and  collaboration  in  Web  authoring. 


Finding 

After  taking  the  GEFT,  ten  out  of  the  twenty-eight  volunteers  were  identified  as  field  independent  learners, 
sixteen  out  of  the  twenty-eight  were  field  dependent  learners,  and  two  participants  were  field  neutral.  For  the 
research  purpose,  the  researcher  did  not  analyze  the  data  collected  from  the  field  neutral  participants. 

Students  in  the  Web  authoring  classes  had  choices  to  design  and  develop  their  Web  projects  individually  or 
with  a group  of  two  members.  Fourteen  participants  in  the  study  chose  to  do  their  Web  authoring  projects 
individually,  and  twelve  participants  worked  on  their  projects  in  groups. 

Participants  were  not  aware  of  their  cognitive  styles  in  field  dependency  or  field  independency  before  they 
chose  to  do  the  Web  authoring  project  individually  or  cooperatively.  In  the  study,  among  the  fourteen  participants 
chose  to  do  the  project  individually,  eight  of  them  are  field  independent  learners  and  six  are  field  dependent  learners. 
On  the  other  hands,  ten  out  of  the  twelve  participants  chose  to  do  the  project  cooperatively  were  field  dependent 
learners  and  only  two  field  independent  participants  chose  to  do  Web  projects  individually. 

Effects  of  Cooperative/Individual  Learning 

It  was  found  that  participants  who  designed  and  developed  their  Web  projects  with  a partner  had  a higher 
mean  score  than  participants  who  designed  and  developed  their  Web  projects  individually.  Participants  who 
completed  their  Web  projects  with  a partner  received  a mean  score  of  33.67  with  a maximum  score  of  35;  however, 
participants  who  completed  their  projects  individually  had  a mean  score  of  29.57  (see  Figure  1). 
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Figure  1:  Comparison  of  Mean  Scores  Between  Individual  and  Group  Learning. 


Specific  group  statistics  was  in  Figure  2. 


variable 

N 

Mean 

Std. 

Deviation 

Std.  Error 
Mean 

SCORES 

Individual 

14 

29.4286 

3.2071 

1.2122 

Group 

12 

33.6667 

.5164 

.2108 

Figure  2:  Group  Statistics  on  Individual  and  Group  Learning 
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The  researcher  used  T-test  to  analyze  the  data,  and  found  a significant  difference  between  individual 
learning  and  group  learning  in  a project-based  learning  environment  with  the  alpha  at  .05  level.  Specific  data  was 
displayed  at  Figure  3. 


t 

df 
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Difference 
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Difference 
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Confidence 
Interval  of 
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assumed 

-3.182 

11 

.009 

-4.2381 

1.3319 

-7.1697 

-1.3065 

Equal 

variances 

not 

assumed 

-3.445 

6.361 

.013 

-4.2381 

1 .2304 

-7.2077 

-1.2685 

Figure  3:  Independent  Samples  Test  on  Group/Individual  Learning 

Effects  of  Learners’  Field  Dependence/Independence 

The  study  showed  that  field  dependent  learners  had  a higher  mean  score  than  field  independent  learners 
when  designed  and  developed  Web  pages  with  project-based  learning  approach.  Field  dependent  learners  had  a 
mean  score  of  32.375  with  a maximum  score  of  35,  and  field  independent  learners  have  a mean  score  of  30  (see 
Figure  4). 


Figure  4:  Comparison  of  Mean  Scores  Between  Field  Dependent  (FD)  and  Field  Independent  (FID)  Learners. 
Specific  group  statistical  data  was  in  Figure  5. 


Learning 

Styles 

N 

Mean 

Std. 

Deviation 

Std.  Error 
Mean 

Scores 

FD 

16 

32.375C 

3.0677 

1.0846 

FI 

10 

29.800C 

2.9496 

1.3191 

Figure  5:  Group  statistics  on  FD  and  FID  Learners. 

The  researcher  again  applied  T-test  to  investigate  whether  there  was  a significant  difference  between 
learners’  field  dependency  and  field  independency  in  a project-based  learning  environment  on  Web  authoring.  No 
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significant  difference  was  found  between  group  learning  and  individual  learning  with  the  alpha  level  at  .05  (Figure 

6). 
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d1 
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tailed) 
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Difference 

Std.  Error 
Difference 

95% 
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Difference 
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1.493 

11 

.164 

2.5750 

1 .7247 

-1.2210 

6.3710 

If 

1.508 

8.910 

.166 

2.5750 

1 .7077 

-1.2941 

6.4441 

Figure  6:  Independent  samples  Test  on  the  effects  of  group  learning  and  individual  learning 

Interactive  Effects  of  Cooperative  Learning  and  Learners’  Learning  Styles 

To  investigate  the  interaction  effects  between  factors  of  cooperative  learning,  individual  learning,  field 
dependent  learning  and  field  independent  learning,  the  researcher  applied  Univariate  Analysis  of  Variances.  It  was 
found  that  there  was  no  significant  effect  between  these  factors  with  alpha  level  at  .05.  Specific  between-subjects 
effects  were  in  Figure  7. 
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1.817 
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34.116 
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34.116 
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1 

2.243E-02 

.003 

.955 

Figure  7:  Tests  of  Between -Subjects  Effects 


Discussions 

Discussions  of  the  research  were  based  on  the  three  major  research  questions  proposed  in  the  study.  First, 
what  influences  do  collaboration  and  individualization  have  on  students’  learning  of  Web  page  development  in  a 
project-based  learning  environment?  The  study  found  that  learners  would  have  a better  academic  performance  if  the 
collaboration  strategy  was  applied  in  a project-based  learning  environment  with  the  learning  task  on  the 
development  of  Web  pages. 

Second,  how  does  students’  learning  style,  specifically  field  dependence/independence,  affect  their  learning 
of  Web  page  development  in  a project-based  learning  environment?  According  to  the  statistical  results,  there  was 
no  significant  difference  between  field  dependent  learners  and  field  independent  learners  when  learning  to  design 
and  develop  Web  pages  with  a project-based  learning  approach.  However,  if  comparing  the  mean  scores  of  both 
groups  of  learners,  it  was  found  that  field  dependent  learners  performed  better  in  developing  Web  authoring  projects 
than  field  independent  learners.  Findings  in  the  study  seems  contradictory  to  the  literature  review  which  suggested 
field  independent  learners  are  more  likely  to  have  an  internal  locus  of  control,  are  more  efficient  information 
processors,  and  usually  perform  better  in  academic  learning  (Davis,  1991;  Hall,  2000).  In  the  study,  most  of  the 
field  dependent  learners  completed  their  Web  authoring  projects  with  a partner.  Only  two  field  dependent  learners 
chose  to  complete  their  projects  individually.  However,  it  is  very  interesting  to  find  that  all  of  the  field  independent 
learners  chose  to  complete  their  projects  individually.  Therefore,  it  was  concluded  that  field  dependent  learners 
could  learn  better  with  group  learning  in  a project-based  learning  approach  of  Web  authoring. 

Third,  what  is  the  relationship  between  instructional  strategies  of  collaboration  and  individualization  and 
learners’  learning  styles  in  field -dependency  and  field -independency  in  a project-based  learning  environment  on 
Web  page  development?  No  relationship  was  found  between  these  factors;  however,  it  was  found  that  most  of  the 
field  dependent  learners  (83%)  chose  to  do  the  project  cooperatively,  and  only  17%  of  field -independent  participants 


chose  to  do  the  Web  project  individually.  This  finding  reflected  the  theoretical  assumption  suggested  by  Leventhal 
and  Sisco  (1996)  that  “field  dependents  are  more  likely  to  be  successful  self-monitors  in  a social  group,”  and  by 
Jones  (1993)  that  field  dependent  learners  “prefer  feedback  and  social  sources  of  information.”  On  the  other  hand, 
Jones  (1993)  also  suggested  that  “field  independent  learners  are  more  individualistic  and  rule -oriented  and  less 
likely  to  seek  peer  input. 


Conclusions 

Several  interesting  findings  were  found  in  the  study.  First,  field  dependent  learners  tended  to  choose  to  do 
the  project  cooperatively,  and  field -independent  participants  tended  to  choose  to  do  the  Web  project  individually. 
Second,  field  dependent  learners  performed  better  in  developing  Web  authoring  projects  than  field  independent 
learners  when  group  learning  strategy  was  used  in  a project-based  learning  approach  of  Web  authoring. 

In  conclusion,  the  study  found  that  field  dependent  learners  have  a better  academic  performance  than  field 
independent  learners  if  they  work  cooperatively  in  a project-based  learning  environment.  By  understanding  the 
relationship  of  learners’  field  independent/dependent  cognitive  style,  cooperative  learning,  and  project-based 
learning,  teachers  can  better  design  an  effective  instructional  environment  for  their  students. 
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A Rubric  for  Evaluating  Final  Group  Multimedia  Projects 


Beginning 

Developing 

Accomplished 

Score 

Score 

0-1  point 

2-3  points 

4 points 

Content 

Organization 

The  contents  are  not 
organized,  and  the  reader 
will  get  lost. 

The  contents  are  partially 
organized,  and  the  reader 
sometimes  can  navigate 
be  themselves;  however, 
sometimes  get  lost. 

The  contents  are  organized, 
and  easy  for  its  readers  to 
go  through  the  project.  A 
site  map  was  provided. 

Interface  design 

The  interface  design  was 
not  based  on  any  human 
computer  interaction 
theories.  Readers  have 
difficulty  understand  the 
information  being 
displayed. 

The  interface  design  was 
based  on  a few  human 
computer  interaction 
theories.  Readers  can 
understand  the 
information  being 
displayed. 

The  interface  design  was 
based  on  sounded  human 
computer  interaction 
theories.  Readers  can 
understand  the  information 
being  displayed  and  easy  to 
follow. 

Interactivity 

There  is  no  interactivity 
between  users  and  the 
project. 

There  are  only  low  levels 
of  interactivities  in  the 
project. 

There  are  high  levels  of 
interactivities  in  the 
project. 

Resources 

Resources  provided  are 
not  sufficient  for  students 
to  accomplish  the  task. 
OR 

There  are  too  many 
resources  for  learners  to 
look  at  in  a reasonable 
time. 

There  is  some  connection 
between  the  resources  and 
the  information  needed 
for  students  to  accomplish 
the  task.  Some  resources 
don't  add  anything  new. 

There  is  a clear  and 
meaningful  connection 
between  all  the  resources 
and  the  information  needed 
for  students  to  accomplish 
the  task.  Every  resource 
carries  its  weight. 

Overall  Visual 
Appeal 

There  are  few  or  no 
graphic  elements.  No 
variation  in  layout  or 
typography. 

Or 

Color  is  garish  and/or 
typographic  variations  are 
overused  and  legibility 
suffers.  Background 
interferes  with  the 
readability. 

Graphic  elements 
sometimes,  but  not 
always,  contribute  to  the 
understanding  of 
concepts,  ideas  and 
relationships.  There  is 
some  variation  in  type 
size,  color,  and  layout. 

Appropriate  and  thematic 
graphic  elements  are  used 
to  make  visual  connections 
that  contribute  to  the 
understanding  of  concepts, 
ideas  and  relationships. 
Differences  in  type  size 
and/or  color  are  used  well 
and  consistently. 

Navigation  & 
Flow 

Getting  through  the  lesson 
is  confusing  and 
unconventional.  Pages 
can't  be  found  easily 
and/or  the  way  back  isn't 
clear. 

There  are  a few  places 
where  the  learner  can  get 
lost  and  not  know  where 
to  go  next. 

Navigation  is  seamless.  It 
is  always  clear  to  the 
learner  what  all  the  pieces 
are  and  how  to  get  to  them. 
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Mechanical 

Aspects 

There  are  more  than  5 
broken  links,  misplaced  or 
missing  images,  badly 
sized  tables,  misspellings 
and/or  grammatical 
errors. 

There  are  some  broken 
links,  misplaced  or 
missing  images,  badly 
sized  tables,  misspellings 
and/or  grammatical 
errors. 

No  mechanical  problems 
noted. 

Total  Score 

/35 
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Abstract 

This  paper  explains  how  handheld,  wireless  computers  were  incorporated  into  the  design  of  an  interactive 
face-to-face  classroom  instruction  and  reports  the  effects  of  such  design  (continuous  assessment  and  immediate 
feedback)  on  the  quality  of  instruction  and  student  learning  in  a higher  educational  institution.  The  design  and 
development  specifications  are  explained  in  detail.  The  evaluation  results  revealed  that  providing  continuous 
assessment  and  immediate  feedback  using  handheld,  wireless  computers  during  instruction  had  a positive  effect  on 
students  ’ knowledge  of  their  own  learning  and  attitude  toward  various  forms  of  assessment  and  the  usage  of 
handheld  computers.  The  result  of  data  analysis  also  indicated  that  using  handheld  computers  to  continuously 
assess  student  learning  followed  by  immediate  feedback  enhanced  the  level  of  class  participation  and  interactions 
among  students  and  between  students  and  the  instructor. 

Introduction 

Assessment  of  student  learning  is  a critical  component  of  any  instructional  system  (Bloom,  1984;  Gagne, 
Wager  & Briggs,  1992;  Gagne,  1985).  Research  indicates  that  continuous  assessment  serves  two  strongly 
interrelated  purposes:  (1)  to  increase  student  learning  (Crooks,  1988)  and  meet  learning  outcomes  (Black  & Wiliam, 
1988;  Gibbs  1992;  Rowntree  1977;  Ramsden  1992),  and  (2)  to  increase  motivation  and  interaction  (Cornel  & 

Martin,  1997;  Crooks,  1988;  Moore  & Kearsley,  1996).  Students  learn  more  in  classes  that  use  frequent  assessment 
than  in  those  that  do  not,  and  brief,  frequent  assessments  are  more  effective  than  infrequent  ones  (Bangert -Drowns, 
Kulik,  & Kulik,  1988,  Dempster,  1991).  Knowledge  of  results  is  also  critical  for  all  learners.  As  assessment  is  a 
vehicle  for  providing  this  knowledge,  research  results  also  emphasize  that  frequent  and  meaningful  assessment  has  a 
powerful  effect  on  students’  overall  motivation  and  learning  (e.g.,  Covington,  1992;  Angelo  & Cross,  1993;  Black  & 
Wiliam,  1998;  Mason  & Woit,  1999). 

Assessment  data  are  also  useful  in  determining  and  improving  the  quality  of  instruction  (Pilcher,  2001).  A 
continuous  or  frequent  assessment  allows  a teacher  to  confirm  students’  understanding  of  the  material  being 
introduced,  to  diagnose  learning  problems,  and  to  provide  remediation  before  proceeding  into  the  next  section. 
Continuously  providing  assessments  throughout  instruction  not  only  offers  teachers  an  opportunity  to  observe  the 
change  and  progress  of  students’  performance,  but  such  assessments  also  help  students  to  consolidate  their  learning 
during  the  process  of  knowledge  construction.  Moreover,  the  feedback  that  teachers  give  as  a result  of  continuous 
assessment  provides  the  scaffolding  that  allows  learners’  understanding  of  how  to  grow  and  evolve  (Cornel  & 

Martin,  1997;  Moore  & Kearsley,  1996;  Schnorr,  & Hazari,  1999). 

Electronic  technology  plays  an  important  role  in  assessment.  Technology  can  be  used  for  assessment 
purposes  at  various  levels  ranging  from  managing  the  assessment  information  to  providing  a fully  automated 
assessment  system.  Researchers  who  studied  the  effects  of  computers  on  student  learning  processes  claimed  that  the 
computer-based  assessment  system  improves  the  quality  of  student  learning  (e.g.,  Oswald,  1996;  Proctor  & 
Donoghue,  1994).  It  is  argued  that  such  an  assessment  system  enables  improvements  in  the  quality  of  learning  by 
ensuring  objectives  and  reproducible  assessment  of  learning  outcomes,  providing  immediate  feedback,  saving  time 
in  monitoring  student  progress,  and  freeing  up  time  for  teachers  to  assess  the  quality  of  their  courses  by  quantitative 
analysis  of  learning  outcomes.  Until  recently,  individual  students  did  not  have  full  access  to  computers  during 
instruction  in  traditional  classroom  settings  (unless  they  were  in  the  computer  laboratories).  Handheld  computers, 
with  their  low  cost  and  high  portability  provide  great  promise  for  individual  student’s  access  to  computer  technology 
during  instruction  and  in  the  traditional  classroom  settings. 

The  advancement  of  handheld  computer  technology  offers  various  functions  and  tools,  allows  students  to 
respond  instantly  to  the  instructor’s  inquiries,  permits  the  instructor  to  assess  students’  responses  and  to  offer 
individualized  point-of-need  feedback.  The  handheld  computers  are  also  capable  of  storing  and  managing 
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assessment  data.  This  capability  reduces  a great  deal  of  instructors’  chores.  Through  the  employment  of  this 
technology,  an  innovative  assessment  method,  such  as  continuously  providing  assessments  and  personalized 
immediate  feedback  in  order  to  promote  teaching  and  learning  in  a traditional  classroom,  becomes  possible. 

The  purpose  of  this  paper  is  to  explain  how  handheld,  wireless  computers  were  used  to  design  and  develop 
interactive  face-to-face  instruction  and  to  provide  some  evidence  of  the  effects  of  such  design  (continuous 
assessment  and  immediate  and  delayed  feedback)  on  the  quality  of  instruction  and  student  learning  in  a higher 
educational  institution.  The  paper  specifically  attempts  to  answer  the  following  questions:  How  and  in  what  ways 
can  handheld,  wireless  computers  be  used  to  enhance  student  learning  and  improve  student  attitude  and  motivation?; 
and  How  and  in  what  ways  the  use  of  handheld,  wireless  computers  influence  the  design  and  quality  of  a face-to- 
face  instruction? 


Methodology 

A total  of  52  students  enrolled  in  one  of  the  three  education  courses  (early  childhood,  middle  school 
education,  special  education)  participated  in  this  project.  All  three  courses  are  required  courses  for  undergraduate 
students  seeking  a baccalaureate  degree  in  education  and  teaching  licensure.  At  least  one  unit  from  each  course  was 
selected  to  integrate  the  use  of  handheld,  wireless  computers  in  its  design,  development,  and  evaluation.  A unit  was 
defined  as  a minimum  of  two  one-hour  and  fifteen  minutes  classes. 

Design  specifications  for  each  course  and  unit  of  instruction 

The  following  common  features  and  design  specifications  were  used  for  each  course  and  its  units  of 
instruction,  regardless  of  contents  and  targeted  learning  outcomes: 

Continuous  assessment  strategies  - This  design  feature  assumed  that  assessment,  which  enhances 
learning,  incorporates  activities  that  are  consistent  with,  and  sometimes  the  same  as,  the  activities  used  in 
instruction.  Such  ongoing  assessment,  therefore,  encourages  and  supports  further  learning  by  documenting 
development  of  the  knowledge,  skills,  and  dispositions  of  learners.  Continuous  assessment  of  students'  thoughts  and 
work  not  only  facilitates  their  learning  but  also  enhances  their  confidence  in  what  they  understand  and  can 
communicate.  When  done  properly,  such  assessment  also  helps  teachers  adjust  their  instruction.  Given  the  above 
assumptions,  the  following  design  specifications  were  integrated  in  each  unit. 

• Ungraded  team  activities  were  built  into  study  materials  (e.g.,  real  world  problem-solving  tasks  or  cases). 

• Short  self-assessment  quizzes  that  allow  learners  to  check  their  own  learning  at  the  beginning  and/or  at  the 
end  of  the  unit  were  designed.  The  quizzes  were  developed  using  quiz  maker  software  that  was  available  on 
campus  through  the  WebCT  course  management  system. 

• Graded  individual  assignments  that  were  specific  to  the  content  of  the  unit  and  assessed  students’ 
understanding  of  the  reading  materials  were  also  incorporated  into  the  unit.  The  expectation  was  that 
students  would  submit  this  assignment  using  the  Internet  and  handheld  computers  at  the  beginning  of  each 
unit.  Formal  and  timely  feedback  was  then  to  be  provided  by  the  instructors  and  peers. 

Timely  and  formative  feedback  - The  second  design  feature  was  the  use  of  timely  feedback.  Research 
shows  that  feedback  on  student  performance  is  a critical  part  of  formative  assessment.  It  is  critical  to  the  quality  of 
learning,  and  can  have  a very  positive  effect  on  the  learning  process.  Feedback  for  assessment  comes  in  many  forms 
(written/verbal,  formal/informal,  and  group/individual),  and  various  degrees  of  usefulness.  To  facilitate  learning,  it 
was  assumed  that  feedback  should  be  informative,  meaningful  and  constructive.  Students  should  understand  whether 
or  not  they  have  achieved,  and  to  what  extent  they  have  achieved,  the  assessment  criteria,  and  also  what  further 
action  is  required  for  better  performance. 

Each  unit  was  then  re-designed  in  a way  that  students  either  received  automated  feedback  on  their 
assignments  (quizzes,  graded  individual  assignment,  ungraded  team  activities,  etc.)  or  received  feedback  from  the 
instructors.  The  WebCT  course  management  system  was  used  to  develop  interactive  tests  with  a wide  array  of 
functions  to  provide  automated  feedback.  The  instructors  also  provided  timely  and  prompt  scaffolding  feedback  for 
both  individual  assignments  and  team  activities. 

Immediate  feedback  and  interactive  learning  environment  using  Student  Response  System  (SRS)  - 

High  levels  of  interactivity  are  essential  to  instructional  systems  (Moore  & Kearsley,  1995;  Muirhead,  1999;  Zirkin 
& Sumler,  1995).  Meaningful  learning  occurs  only  when  students  actively  participate  in  the  learning  process. 
Learner  feedback  is  essential  in  determining  students’  needs,  in  measuring  comprehension,  in  encouraging  their 
interest  and  participation,  and  in  improving  learning.  However,  even  skillful  teachers  find  it  difficult  to  involve  all 
the  students  in  the  large  group  classroom  discussion  simultaneously.  In  this  study,  a Student  Response  System 
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(SRS)  was  used  for  each  unit  to  allow  instructors  to  ask  questions  during  lectures  and  large  group  discussions  and  to 
receive  immediate  responses  from  all  students. 

Numina  II  SRS  is  a Web-based  polling  system  developed  by  UNCW  Computer  Science  Department.  It 
uses  a combination  of  wireless  networks,  handheld  computers,  and  a data  projector  to  allow  students  to  submit  their 
responses  to  questions  posed  by  the  instructor  and  to  immediately  display  the  results  of  submission  using  bar  graph 
for  quantitative  data  and  text  for  comments.  With  this  manner,  an  instructor  is  able  to  measure  students’ 
comprehension  or  opinion  on  a posed  question  and  to  provide  just-in-time  remediation  whenever  necessary.  By 
using  the  handheld  computers  and  SRS,  students  are  able  to  respond  to  questions  without  having  to  raise  their  hands 
and  risk  exposing  their  lack  of  understanding,  and  the  instructor  can  diagnose  misconceptions  instantly  from  the 
submitted  responses  that  are  displayed  in  graphical  format  for  the  entire  class. 

In  preparation  for  the  use  of  this  system  during  the  live  instruction  for  each  unit,  the  instructors  developed 
key  questions  related  to  the  concepts  covered  in  their  lecture  or  large  group  discussions.  The  questions  then  were 
posed  to  students  during  the  instructor’s  lecture  or  class  discussion  in  order  to  receive  feedback  for  immediate 
remediation  of  instruction  and  discussion.  The  nature  and  the  format  of  the  questions  differed  depending  on  the 
content  of  the  unit,  expected  outcomes,  and  the  instructor’s  teaching  styles. 

Implementati  on  Procedure 

The  instructors  and  the  researchers  collaborated  in  designing  and  developing  the  instructional  materials  for  each  unit 
while  the  instructor  for  each  course  was  solely  responsible  for  providing  the  content.  The  researchers  also 
developed  a try-out  session  in  order  to  train  students  and  familiarize  them  with  the  implementation  procedure.  The 
try-out  session  emulated  the  implementation  procedures  and  trained  students  in  the  use  of  handheld  computers  for 
both  taking  online  quizzes  and  assignments  and  responding  to  SRS.  All  instruction  took  place  in  classrooms  that  had 
wireless  connection  to  the  Internet.  A set  of  handheld  PCs  (HP  Jornada)  that  came  standard  with  the  pocket  version 
of  Microsoft  Office  (Word,  Excel,  Access,  and  PowerPoint)  and  a WebBrowser  and  were  equipped  with  wireless 
network  cards  was  used  in  both  courses.  The  following  procedure  summarizes  the  implementation  process. 

• Students  received  notification  from  the  instructor  to  complete  the  unit’s  individual  assignment  before  the 
class  session  by  logging  on  to  the  WebCT  course  site  and  completing  their  individual  assignment.  The 
instructor  then  skimmed  over  students’  responses  to  the  individual  assignment  before  the  first  class  session. 

• At  the  beginning  of  the  first  class  session,  handheld  computers  were  distributed  to  students.  Students  then 
were  asked  to  connect  to  the  Internet  and  log  on  to  the  WebCT  site  and  complete  a pretest  (quiz)  that 
covered  the  objectives  of  the  unit.  Students  were  also  encouraged  to  view  the  results  of  their  test  if  the 
items  were  closed  ended.  However,  no  formative  feedback  was  provided  during  the  pretest. 

• Upon  completion  of  the  pretest  (quiz)  the  first  class  session  proceeded  with  the  instructor’s  lecture  and 
large  group  discussion.  The  instructor  used  the  SRS  during  his/her  lecture  to  pose  questions  and  receive 
feedback  from  students.  The  results  of  students’  responses  were  projected  as  soon  as  they  were  submitted, 
both  quantitatively  (graphs)  and  qualitatively  (students’  narrative  memos  to  support  the  yes/  no, 
agree/disagree,  true/false  or  multiple  choice  questions)  and  were  followed  by  large  group  discussion. 

• After  the  first  but  before  the  next  class  session,  the  instructor  reviewed  students’  responses  to  the  individual 
assignment  more  carefully  and  provided  individual  and  formative  feedback  to  students  (through  e-mail  or 
WebCT  course  management  system). 

• The  second  class  session  began  by  distributing  handheld  computers  to  students  and  asking  them  to  open  a 
word  file  that  contained  the  team  activity,  which  was  saved  on  the  desktop  of  their  handheld  computers, 
and  reviewing  the  content  of  the  team  activity.  The  instructor  then  provided  a brief  overview  of  the 
previous  session’s  discussions  and  explained  the  team  activity  and  its  purposes.  Students  were  then  grouped 
into  teams  to  discuss  the  team  activity  and  to  compose  a short  report.  The  recorder  of  each  team  was 
expected  to  send  his/her  team’s  report  to  the  instructor  via  e-mail  and  beam  it  (transmitting  data  wireless 
via  Infrared)  to  the  other  members  of  the  group.  Respectively,  the  instructor  used  the  wireless  system  to 
receive  the  teams’  responses/reports  as  they  were  submitted  by  each  team  through  e-mail,  compile  them  in 

a word  file,  and  project  them  for  the  large  group  discussion. 

• At  the  end  of  the  second  session,  students  were  given  time  to  log  on  to  the  Internet  using  their  handheld 
computers  and  complete  the  posttest.  Students  were  encouraged  to  view  the  results  and  the  feedback  for 
both  close-ended  and  open-ended  test  items.  They  were  also  asked  to  complete  a print-based  anonymous 
attitude  survey. 
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Evaluation  and  data  analysis  strategies 

Data  was  collected  from  the  following  sources:  (1)  students’  performance  on  quizzes  (pre-  and  posttest 
consisting  of  both  multiple -choice  and  short  answer  items),  individual  and  team  activities  using  records  generated 
and  kept  in  handheld  computers;  (2)  students’  responses  to  an  attitude  survey  (using  5-point  scale  with  5=strongly 
agree,  and  l=strongly  disagree)  which  measured  students’  attitude  toward  different  forms  of  assessment  and  the 
usage  of  handheld  computers;  (3)  instructors’  observational  notes  on  the  students’  level  of  engagement  in  class 
activities  and  interactions.  A combination  of  quantitative  and  qualitative  data  analysis  approach  was  used  to  make 
sense  of  the  data. 

Results  and  Discussions 

Three  independent  Paired  Samples  t-tests  were  used  to  analyze  the  students’  performance  on  quizzes  for 
each  class.  The  analysis  results  yielded  significant  differences  between  pre-  and  posttest  scores  across  all  three 
groups  (t  = 9.95,  df=19,  p<.001;  t=15.8,  df=19,  p<.001;  t=5.3,  df=8,  p<.001).  The  integration  of  continuous 
assessment  and  immediate  feedback  into  the  design  of  instructional  material  and  activities  delivered  by  handheld, 
wireless  computers  may  account  for  the  significant  improvement  of  the  students’  performance  on  the  posttest. 


Table  1:  The  results  oft-test  analyses  of  students’  performance  on  quizzes 


Courses 

N 

Mean 

T-Test 

P 

Pretest 

Posttest 

Early  Childhood  Education 

10 

27 

39 

5.30 

<.001 

Special  Education 

21 

32 

57 

9.95 

<.001 

Middle  School  Education 

21 

.4 

6.5 

15.8 

<.001 

The  analysis  of  the  students’  attitudes  survey  toward  different  forms  of  assessment  and  the  usage  of 
handheld  computers  showed  that  students  thought  the  use  of  various  assessment  strategies  (M=4.4)  followed  by 
immediate  feedback  (M=4.5)  during  instruction  had  a positive  effect  on  their  knowledge  of  their  own  learning.  The 
analysis  further  revealed  that  students  found  the  SRS’  bar  graph  display  of  the  student  responses  was  a useful 
feedback  affecting  their  performance  (M=4.5).  The  same  rating  was  also  indicated  in  the  category  of  immediate 
feedback  on  posttests  (M=4.5).  Students  also  reported  that  the  incorporation  of  the  handheld  wireless  computers 
enhanced  their  level  of  involvement  (M=4.1)  created  more  opportunities  for  interactions  both  amongst  students  and 
between  students  and  the  instructor  (M=4.1),  and  made  the  unit  of  instruction  more  interesting.  Overall,  students  felt 
positive  about  using  handheld  computers  during  classroom  instruction.  They  indicated  that  they  enjoyed  having  an 
opportunity  to  use  this  new  technology  and  thought  that  they  learned  how  this  technology  could  enhance  student 
learning  in  the  classroom. 

The  analysis  of  instructors’  observational  notes  on  the  level  of  student  engagement  in  team  activities 
suggested  that  students  were  actively  involved  in  the  team  discussions  and  appeared  to  enjoy  the  process  of  sharing 
files  and  ideas.  The  instructors’  notes  indicated  that  using  handheld  computers  and  the  immediate  feedback  from 
students  helped  in  aligning  instructional  objectives  with  instructional  content  and  evaluation,  and  in  monitoring 
students’  learning  and  understanding  their  perceptions  and/or  misconceptions.  The  instructor  also  thought  that 
having  access  to  students’  understanding  of  the  concepts  during  instruction  assisted  in  tailoring  the  instruction 
toward  students’  needs.  Overall,  it  appeared  that  focusing  on  designing  instruction  for  continuous  assessment  of 
student  learning  using  handheld  computers  changed  the  class  dynamic  from  a teacher-directed,  lecture  driven 
classroom  to  a more  student-centered  and  student-controlled  learning  environment. 
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Abstract 

This  study  examines  the  process  of  employing  technology  to  support  in-service  teachers  in  the  development 
of  science  curriculum.  The  action  research  concludes  with  four  main  challenges  in  the  integration  of  technology  into 
curriculum:  computer  access,  computer  experience,  technology  support  and  institutional  support.  This  study 
suggests  various  ways  to  strengthen  technical  support  such  as  electronic  portfolio,  site  visit,  tutorials,  and  computer 
software.  Nevertheless,  access  to  computer  remains  a critical  issue  for  in-service  teachers  in  rural  areas. 


Introduction 

As  technology  has  become  ubiquitous  in  K-12  environments,  the  successful  integration  of  technology  into 
curriculum  and  instruction  remains  a main  challenge  in  teacher  education.  The  lack  of  sound  pedagogy  and 
instructional  support  to  integrate  technology  into  instruction  can  account  for  some  of  the  resistance  among  teachers 
to  hop  on  the  bandwagon  of  technology.  Many  teachers  are  novice  technology  users  and  there  are  not  enough 
opportunities  for  professional  development.  This  study  aims  to  evaluate  the  process  of  providing  technological 
support  to  K-12  teachers  and  examine  factors  contributing  to  or  impeding  the  integration  of  technology  into 
curriculum  in  a teacher  education  course  at  the  College  of  Education.  The  constructivist  approach  that  emphasizes 
learner-control  and  co-construction  of  knowledge  in  a real-world  context  is  adapted  to  facilitate  the  process  of 
providing  technology  integration.  Action  research  that  investigates  the  means  to  instructional  improvement  within 
the  educational  experience  is  employed  for  this  study. 

This  research  focuses  on  the  process  of  implementing  technological  support  to  a group  of  science  teachers 
with  different  levels  of  technical  skills.  The  objectives  of  this  study  are  to: 

a.  Determine  how  technology  can  provide  support  for  curriculum  development 

b.  Evaluate  the  constructivist  approach  to  the  integration  of  technology  into  curriculum  design 

Theoretical  Framework 

Piaget  and  Vygotsky  are  two  thinkers  whose  impact  on  constructivism  is  profound.  Constructivist  theorists 
who  draw  from  Piaget  put  more  emphasis  on  individual  construction  of  knowledge  as  a result  of  interaction  with  the 
physical  environment.  Constructivist  theorists  who  are  influenced  by  Vygotsky  posit  that  knowledge  is  constructed 
through  the  appropriation  of  culturally  relevant  activities.  In  other  words,  knowledge  is  co-constructed  with  peers  or 
experts  and  through  immersion  in  a social  context  (Bonk  & Cunningham,  1998).  Jonassen  et  al.  (1995),  long-time 
advocates  of  constructivism  for  distance  education,  argue: 

Constructivist  principles  provide  a set  of  guiding  principles  to  help  designers  and  teachers  create  learner- 
centered,  collaborative  environments  that  support  reflective  and  experiential  processes.  Students  and  instructors  can 
then  build  meaning,  understanding,  and  relevant  practice  together  and  go  far  beyond  the  mere  movement  of 
information  from  instructors’  minds  to  students’  notebooks,  (p.  8) 

According  to  Jonassen  et  al.  (1995),  the  four  constructivist  attributes  for  building  learning  systems  are 
context,  construction,  collaboration,  and  conversation.  Context  refers  to  the  “real  world’’  scenario  in  which  learners 
can  carry  out  learning  tasks  as  close  to  the  real  world  as  possible.  Learning  tasks  should  have  real-world 
implications  so  that  learners  can  connect  what  they  learn  in  the  classroom  with  the  real  world.  Construction 
concerns  knowledge  that  is  built  on  the  “active  process  of  articulation  and  reflection  within  a context’’  (Jonassen, 
1995,  p.  8).  Learners  acquire  knowledge  better  when  they  can  link  their  own  experience  with  the  learning  materials 
and  make  sense  of  them.  Learners  master  a subject  better  in  the  process  of  constructing  knowledge.  Collaboration 
helps  learners  to  develop,  test,  and  evaluate  their  ideas  with  peers.  Learners  are  exposed  to  multiple  perspectives  in  a 
problem-solving  case  and  then  come  to  a self -selected  conclusion  on  a particular  issue.  This  is  an  important  part  of 
the  learning  process.  Conversation  is  engaged  in  by  group  members  for  purposes  such  as  planning,  collaboration. 
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and  meaning  making.  It  is  especially  important  for  distance  learning  because  most  communication  is  done  through 
online  exchanges.  A successful  conversation  will  lead  to  good  preparations  for  and  completion  of  online  tasks 
(Jonassen  et  al,  1 995).  These  four  attributes  also  serve  as  the  framework  for  the  implementation  of  technology  in  the 
course  for  this  study. 


Course  Description 

Background 

The  course  for  this  research  is  titled  "Interdisciplinary  Science  Curriculum:  Malama  I Ka  Aina, 
Sustainability  (subtitle:  Environmental  Bioremediation:  Concepts  and  Practices  in  Environmental-  and  Agriculture- 
based  Science)”.  This  course  is  part  of  the  Department  of  Education  funded  project,  Malama  I Ka  Aina,  in  support  of 
K-12  science  teachers  to  develop  both  standards  based  and  culturally  relevant  science  curriculum  to  Hawaii  students 
as  described  in  the  Malama  web  site  (http://www.hawaii.edu/malama).  Science  teachers  registered  in  this  course 
were  expected  to  achieve  the  main  course  objective:  incorporating  traditional  Hawaiian  cultural  practices  and 
methods  of  modern  environmental  technologies  into  science  curriculum  and  instruction  through  an  immersion 
program  in  the  field  and  hands-on  practice  with  community  experts.  This  course  began  in  summer  2001  and  was 
concluded  in  May  2002. 

Twenty  two  in-service  teachers  from  three  different  Hawaiian  islands  registered  in  the  course  and  eight  site 
teachers  provided  instructional  and  content  support  for  all  teachers  at  different  campuses.  The  course  was  co-taught 
by  two  faculty  members  from  the  College  of  Education  and  College  of  Tropical  Agriculture  and  Human  Resources . 
The  core  instructional  team  consisted  of  the  instructors,  a content  and  curriculum  design  specialist,  and  an 
instructional  designer.  The  instruction  was  supplemented  by  guest  instructors  from  Hawaiian  community  groups. 

The  subject  areas  that  the  participating  teachers  teach  included  biology,  agriculture,  chemistry,  hydroponic,  physics, 
earth  science,  life  science,  marine  science,  environmental  science,  and  general  science.  Since  the  teachers  were  from 
different  islands  and  carry  their  full -load  of  teaching  during  regular  semesters,  meetings  were  usually  scheduled 
during  the  weekends  or  semester  breaks  in  summer.  The  use  of  technology  played  an  important  role  for  continuous 
communication  in  the  year-round  course. 

Standards  -based  Curriculum  Development 

One  important  aspect  of  the  course  is  the  incorporation  of  science  standards  into  unit  and  lesson  plans.  A 
standards -based  curriculum  provides  a set  of  guidelines  for  teachers  to  develop  the  content  and  devise  activities  that 
are  appropriate  for  the  various  levels  of  the  students.  The  standards  employed  in  this  course  are  based  on  the  booklet 
"Hawaii  Content  and  Performance  Standards,  HCPS  11”  developed  by  the  Department  of  Education  at  the  State  of 
Hawaii.  HCPS,  which  provide  specific  benchmarks  and  learning  outcomes  in  K-12  systems,  can  be  divided  into  two 
domains: 

"(1)  Domain  I standards  are  about  science  as  a way  of  thinking  and  knowing.  It  has  roots  in  Scientific 
Inquiry  and  meets  the  first  goal  of  Science  Education  (which  is  understand  and  apply  the  process,  ways  of 
thinking  and  dispositions  that  humans  have  while  investigating  the  Natural  World).  (2)  Domain  II  standards 
are  about  Science  as  Cumulative  Knowledge.  Domain  II  identifies  essential  understandings  that  will  help 
students  meet  the  second  goal  of  Science  Education  (which  is  to  understand  and  apply  the  knowledge  we 
know  today  about  the  world  around  us  to  our  curiosities  and  in  our  daily  lives)"  (p.  2-3,  Office  of 
Accountability  and  School  Instructional  Support/School  Renewal  Group,  1998). 

Examples  on  unit  and  lesson  plans  that  address  the  benchmarks  in  the  standards  can  be  found  at  the  Malama  web 
site. 

Technology 

The  four  constructivist  attributes,  namely,  context,  construction,  collaboration,  and  conversation  provide 
the  foundation  for  the  design  of  the  technological  learning  environment.  The  context  was  the  subject  area  in  which 
the  teachers  developed  the  instruction  and  activities  for  their  own  classroom  use.  This  course  provided  the  training 
opportunities  for  the  teachers  to  acquire  the  skills,  knowledge  and  technology  for  curriculum  development.  They 
were  to  construct  the  curriculum  through  either  group  projects  or  individual  projects  based  on  their  own  disciplines 
and  background  knowledge  in  a subject  area.  Through  collaboration  with  other  teachers  in  the  same  group,  they 
drew  on  experiences  from  their  own  teaching  in  different  grade  levels  and  contributed  their  knowledge  to  the 
project.  Eor  teachers  in  different  groups,  they  engaged  in  online  conversations  to  comment  and  receive  suggestions 
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from  peers  on  their  own  projects.  Throughout  the  entire  year,  online  conversation  was  encouraged  to  increase  the 
understanding  of  the  course  content  and  information  sharing. 

The  role  of  technology  in  the  Malama  program  was  to  foster  a learning  environment  in  which  participants 
collaborated  together  to  develop  a database  of  K-12  standards-based  science  curriculum  for  information 
dissemination.  The  participants  did  not  meet  on  a regular  basis.  The  email  list  and  the  web  tools  played  a vital  role  in 
the  communication  between  instructors  and  participants.  The  Malama  web  site  provided  the  resources  and  venue  for 
teachers  to  disseminate  information,  collaborate  on  projects,  carrying  out  discussion  and  co-construct  knowledge 
together. 

Specifically,  the  tools  employed  in  the  course  included:  email,  web  pages,  bulletin  board,  chat  room,  and 
listserv.  In  order  to  make  sure  that  every  teacher  learned  how  to  use  the  web  editor  Dreamweaver  and  publish  their 
curriculum  on  the  web,  a two -day  technology  workshop  was  planned  and  conducted  by  the  Malama  instructional 
support  group  in  a computer  lab.  The  workshop  description  can  be  found  at 

http://www.hawaii.edu/malama/handouts/DW_workshop.html  (there  is  an  underscore  between  DW  and 
workshop.html).  Participants  were  provided  with  detailed  handouts  and  online  resources  for  learning  about 
Dreamweaver.  Preview  of  the  lesson  plan  and  preparations  for  the  workshop  were  sent  to  the  teachers  in  advance. 
One  of  the  objectives  of  the  workshop  was  provide  teachers  with  the  skills  and  knowledge  necessary  to  maintain 
their  own  web  sites.  Eventually,  they  would  also  be  able  to  integrate  the  technology  into  the  lesson  plans  and  teach 
their  students  the  technical  skills  necessary  to  develop  their  own  projects. 


Research  design  and  data  collection 

Action  research  for  instructional  and  technical  improvements 

Action  research  is  a process  that  involves  all  participants  (e.g.  students,  teachers,  and  other  parties)  in  the 
educational  process  to  work  together  for  the  improvement  of  instruction  and  curriculum.  Social  psychologist  Kurt 
Lewin's  work  (1946)  has  contributed  greatly  to  the  maturation  of  Action  Research  as  a method  for  research. 
According  to  Lewin's  (1946)  definition,  "action  research  is  a three-step  spiral  process  of  (1)  planning  which 
involves  reconnaissance;  (2)  taking  action;  and  (3)  fact-finding  about  the  results  of  the  action"  (p.  27,  cited  by 
Kemmis,  1988).  Action  research  is  ideal  for  generating  insightful  information  for  the  improvement  of  school 
programs.  It  provides  educators  the  opportunities  to  evaluate  new  ideas  about  teaching,  curriculum,  and  learning  so 
that  they  can  make  informative  decisions. 

The  design  of  action  research  is  described  by  Kimmis  & McTaggart  (1988)  in  the  following  cycle:  plan  -> 
act  ->  observe  ->  reflect  ->  and  revise  plan  ->  next  cycle  in  similar  sequence.  This  study  is  conducted  in  the 
following  sequence:  selecting  research  areas/focus,  taking  action,  collecting  data,  organizing  and  interpreting  data, 
reflecting  on  action,  and  revising  action  plan. 

This  research  focuses  on  how  technology  can  support  teachers  in  developing  curriculum.  Two  research 
areas  are  identified:  factors  that  enhance  or  impede  the  support  of  technology  and  assessment  of  the  constructivist 
approach  to  the  implementation  of  technology.  The  research  is  concerned  with  how  technology  can  provide  support 
for  teachers  and  how  teachers  can  benefit  from  the  constructivist  approach.  The  action  plans  include  needs 
assessment,  technology  workshops,  and  follow-up  support.  The  following  data  are  collected  to  assess  the  process 
and  effectiveness  of  the  technological  support:  surveys,  interviews,  observation,  and  project  evaluation. 

Research  areas  This  study  focuses  on  how  technology  can  support  teachers  in  developing  curriculum.  Two 
research  areas  are  identified:  (1)  factors  that  enhance  or  impede  the  support  of  technology,  and  (2)  assessment  of  the 
constructivist  approach  to  the  implementation  of  technology 

For  the  first  area,  the  following  research  questions  investigate  how  technology  can  provide  support  for 
curriculum  development: 

1.  Does  the  use  of  technology  facilitate  curriculum  development? 

2.  Does  technology  provide  the  kind  of  support  for  teachers  to  develop  curriculum? 

3.  What  kind  of  support  can  be  further  provided? 

4.  How  do  novice  teachers  adopt  to  technology?  What  are  the  factors  that  enhance  or  impede  their  adoption 

of  technology? 

For  the  second  area,  the  following  questions  evaluate  the  constructivist  approach  of  integrating  technology 
into  curriculum  design: 

1.  Context  learning:  the  effectiveness  of  using  real-world  context  as  the  backdrop  to  develop  lesson  plans 
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2.  Knowledge  construction:  the  effectiveness  of  knowledge  construction  through  web  resources  and 
information  sharing 

3.  Collaboration:  the  effectiveness  of  team  collaboration 

4.  Conversation:  the  effectiveness  of  peer  critiquing  and  information  sharing  through  the  web  bulletin  board 

Action  Plans  Several  actions  are  taken  to  provide  support  of  technology  to  teachers. 

1.  Needs  assessment:  a preliminary  survey  is  conducted  to  assess  the  technology  literacy  of  each  teacher 
and  the  type  of  technical  support  from  their  respective  schools. 

2.  Tech  workshops:  several  technical  workshops  are  conducted  for  individuals  and  the  whole  class  to 
advance  teachers'  knowledge  in  technology,  specifically  web  page  editing  tools. 

3.  Follow-up  support:  Once  the  teachers'  web  sites  are  uploaded  to  the  Internet,  the  instructional  support 
team  can  monitor  their  progress  and  keep  close  watch  on  problems  that  the  individual  teachers  may 
encounter. 

Data  collection 

The  following  data  are  collected  to  assess  the  process  and  effectiveness  of  the  technological  support: 

a.  Preliminary  survey  on  demographic  background,  technical  skill  levels,  and  the  institutional  support  available. 

b.  Post-workshop  survey  to  evaluate  the  following  aspects  of  the  technology  workshop:  workshop  structure, 
instructor-participants  interaction,  overall  workshop  enjoyment/satisfaction,  and  computer  lab  learning  environment. 
(http://www.hawaii.edu/malama/surveys/workshop_eval.html).  The  questionnaire  is  adapted  from  the  study  by 
Thomerson  and  Smith  (1996). 

c.  Observations  by  instructors:  the  instruction  team  reports  their  observation  on  teacher's  adoption  of  technology 
(http://www.hawaii.edu/malama/surveys/project_observation.html) 

d.  Evaluation  of  final  projects  (e.g.  web  site  completion,  indication  of  employing  technology  in  classroom  activities 
and  student  projects) 

The  data  analysis,  critical  reflection,  and  revised  plan  are  discussed  in  the  next  section. 


Analysis  and  discussion 

Preliminary  study 

There  were  eleven  male  and  eleven  female  in-service  teachers  participating  in  this  course.  The  preliminary 
survey  revealed  four  technological  challenges  in  this  course:  computer  access,  computer  experience,  institutional 
support,  and  technical  support.  As  concerns  computer  access,  three  teachers  did  not  own  a computer  at  home  and 
they  did  not  use  a computer  at  school  for  instructional  purposes.  These  teachers  did  not  have  email  accounts  at  the 
beginning  of  the  course.  This  mixture  of  novice  and  advanced  computer  users  presented  the  first  technical  challenge 
of  this  course:  keeping  the  communication  flow  in  a class  that  did  not  meet  regularly  physically  or  virtually.  The 
instructors  usually  sent  out  class-related  announcement  via  email  and  then  notified  other  teachers  who  did  not  have 
email  accounts  by  phone  calls  or  mail.  After  the  first  technology  workshop  was  held  in  late  July,  all  teachers  except 
for  one  had  e-mail  accounts.  Nevertheless,  the  lack  of  computer  access  for  some  teachers  in  rural  areas  made  it 
impossible  for  them  to  receive  email  promptly  and  efficiently. 

Regarding  computer  experience,  only  two  out  of  twenty-two  in-service  teachers  had  their  own  web  pages. 
The  majority  had  little  or  no  experience  in  creating  web  pages.  The  July  technology  workshop  aimed  at  advancing 
their  web  editing  skills  to  at  least  beginner  level  so  that  they  could  start  planning  curriculum  with  the  aid  of 
technology. 

In  terms  of  institutional  and  technical  support,  although  most  K-12  schools  are  already  equipped  with 
computers,  schools  in  rural  areas  are  still  ill  equipped.  There  is  also  a lack  of  institutional  support  in  providing 
technical  support  for  teachers.  One  of  the  Malama  project  objectives  is  to  empower  the  teachers  with  the  knowledge 
to  develop  web  pages.  This  initiative  was  contingent  upon  the  technical  support  and  Internet  access  from  each  site 
school.  Half  of  the  teachers  were  not  aware  of  the  kind  of  technical  support  that  their  individual  schools  would  have 
provided. 

Technology  workshop  surveys 

The  overall  ratings  of  the  workshop  evaluated  by  the  participants  are  high.  On  a one-to-five  scale,  one  as 
low  and  five  as  high,  the  workshop  ratings  as  indicated  in  the  parentheses  include  the  following  categories: 
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Instructor-participants  interaction  (4.1),  computer  lab  learning  environment  (3.1),  workshop  structure  (4.07),  and 
overall  enjoyment  and  satisfaction  (4.36)  as  shown  in  Table  1.  The  comments  by  the  participants  offered  excellent 
suggestions  in  the  following  areas:  smaller  class  size,  classes  for  participants  with  different  levels  of  computer  skills, 
multiple  sessions,  longer  workshop  hours,  and  more  detailed  instructional  manuals.  Twenty-three  teachers  who 
shared  19  computers  attended  the  two -day  workshop  (three  hours  each  day). 


Table  1:  Post-workshop  Survey  Results 


Instructor  - participants  interaction 

Ql 

I felt  comfortable  asking  the  instructor  questions. 

4.57 

■ 

Q4 

I felt  uncomfortable  asking  for  help  during  workshop 

2.36 

Q5 

The  instructor  was  responsive  to  students'  needs. 

4.36 

Qll 

When  I ask  questions,  the  instructors  give  me  the  answers  I need. 

4.64 

Average 

4.1 

Learning  environment  (computer  lab) 

Q2 

The  overall  workshop  design  was  conducive  to  learning  about  writing  web  pages. 

4 

Q7 

The  screen  layout  and  interface  design  of  portfolio  are  consistent  and  easy  to  use. 

4.14 

■ 

Q13 

I had  a difficult  time  understanding  the  instruction. 

2.86 

■ 

Q16 

I tend  to  get  easily  distracted  in  a computer  training  environment. 

2.79 

Average 

3.1 

Workshop  structure 

Q3 

The  instructor  used  the  computer  effectively  for  meeting  the  objective  of  the  workshop. 

4.43 

Q6 

Examples  and  illustrations  were  effectively  used  by  the  instructor. 

4.21 

Q8 

The  amount  of  material  covered  was  adequate  for  the  length  of  the  workshop. 

3.71 

QIO 

Workshop  content  was  presented  in  a well-organized  manner. 

3.93 

Average 

4.07 

Overall  workshop  enjoyment/satisfaction 

Q9 

I have  a sense  of  accomplishment  so  far. 

4.07 

Q12 

The  method  of  workshop  presentation  kept  my  interest  high  through  the  entire  workshop. 

4.07 

Q14 

I would  recommend  that  other  teachers  take  similar  courses  from  the  same  instructor. 

4.36 

Q15 

The  instructional  team  was  helpful  in  providing  assistance. 

4.93 

Average 

4.36 

*Q4,  Q13,  & Q16  have  been  reversed  for  tabulation. 

Follow-up  support 

Based  on  the  preliminary  surveys,  the  following  action  plans  were  taken  to  provide  continuing  support: 

a.  Schedule  a series  of  workshops  on  computer  literacy,  web  page  design,  image  scanning  and  processing, 
and  power  point  presentation 

b.  Provide  individual  on-site  tutoring  for  teachers  who  need  extra  help 

c.  Communicate  with  school  tech  coordinators  on  the  kind  of  support  and  resources  that  the  teachers  would 
need  in  developing  their  own  web  sites 

d.  Site  visit  with  teachers  who  have  the  least  computer  experience  to  assess  the  kind  of  support  that  was 
needed  for  them  to  utilize  the  technology  for  project  development  at  their  own  schools 

e.  Provide  books  and  computer  software  site  licenses  for  all  teachers  to  develop  individual  projects. 

f Develop  electronic  portfolio  that  provides  standard -based  curriculum  template  for  teachers  to  download 
g.  Establish  a web-based  bulletin  board  on  WebCrossing  for  information  dissemination. 
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Observations 

Two  instructors  and  one  curriculum  specialist  who  were  also  present  to  observe  the  workshop  and  learned 
about  the  technology  made  comments  on  the  following  four  categories:  overall  participant  satisfaction,  course 
structure,  course  design,  and  instructor-participant  interaction.  The  observations  are  summarized  as  below: 

a.  Overall  satisfaction:  While  three  observers  agreed  that  the  members  of  the  workshop  seem  to  be  happy  with 
the  fact  that  they  have  learned  something  new,  the  learners  also  seemed  overwhelmed  by  the  amount  of 
materials  rendered  in  the  workshop.  Because  there  was  a large  variation  in  the  skills  initially,  each 
participant  moved  to  a different  level,  but  all  make  progress  in  their  best  capacity.  One  observer  pointed  out 
that  those  who  had  minimal  exposure  to  computers  would  have  benefited  more  fromone-to-one  tutorial 
session  than  a large  workshop. 

b.  Workshop  structure:  The  course  was  well-organized  and  the  examples  used  are  relevant  to  teachers’ 
background.  While  learning  about  the  technology,  they  also  learned  to  build  an  electronic  portfolio  through 
several  templates.  The  templates  consisted  of  a consistent  layout  and  several  web  pages  that  include  school 
introduction,  project  overview,  standards -based  unit  planning,  unit  and  lesson  plans,  student  works,  and 
photo  gallery.  Participants  can  download  the  electronic  portfolio  templates  and  provide  content  to  the 
templates  without  investing  much  time  on  web  design.  Through  the  use  of  a template,  participants  were  at  a 
place  that  they  could  see  good  quality  results  and  get  excited.  Nevertheless,  observers  suggested  that 
handouts  with  step-by-step  instruction  (where  to  go  or  click  from  point  A to  point  B)  would  be  more  useful. 
The  handout  was  written  for  participants  in  a face-to-face  workshop  for  reviewing  purpose,  not  for  self- 
guided  tutorials.  In  addition,  it  would  be  even  better  if  the  participants  could  upload  their  web  pages  to  the 
server  and  see  the  results.  Since  the  workshop  was  conducted  at  the  beginning  of  the  course,  many 
participants  have  yet  to  received  an  email  account  from  the  University.  Web  server  access  was  tied  to  the 
availability  of  an  active  email  account.  Therefore,  many  of  them  could  not  upload  their  practice  files  to  the 
server  even  though  the  instructions  were  included  in  the  handout. 

c.  Workshop  design:  The  strengths  of  the  workshop  were  the  use  of  the  handouts,  computer  projection,  and  a 
portfolio  that  the  participants  can  download  as  a template  for  their  own  projects.  The  weaknesses  were  the 
large  size  of  class  (19  participants),  fast-paced  instruction,  and  a lack  of  sufficient  time  for  practice.  For 
future  workshops,  an  introductory  course  for  beginners  and  an  intermediate  course  for  participants  with 
basic  skills  would  make  it  easier  to  conduct  the  instruction. 

d.  Instructor-participant  interaction:  While  the  observers  all  agreed  that  the  instructor  was  helpful  and 
responsive  to  questions,  the  use  of  tech  assistants  also  contributed  greatly  to  the  positive  interaction.  It  was 
helpful  for  both  the  instructor  and  the  assistants  traveled  frequently  to  different  corners  of  the  room  to 
provide  timely  assistant  to  the  participants. 

e.  Additional  comments:  Instead  of  compressing  the  instructional  materials  into  two  afternoons,  the 
workshop  could  have  easily  stretched  into  a three-day  workshop  to  allow  more  time  for  practice.  The 
participants  definitely  need  follow-up  support  and  practice  on  their  own  to  retain  all  the  materials  covered 
in  the  workshop. 

Final  project  evaluation 

Fourteen  (64%)  out  of  a total  of  twenty-two  teachers  posted  their  projects  on  the  web.  Teachers  from  the 
same  school  usually  collaborated  on  the  same  project  together.  Half  of  the  projects  posted  on  the  web  showed  a 
good  command  of  basic  web  authoring  skills.  Two  projects  showed  advanced  level  of  web  authoring  mainly  because 
the  two  teachers  had  prior  web  authoring  experience  before  registering  for  the  course.  Two  projects  showed 
intermediate  level  of  web  authoring  skills  and  the  teachers  had  minimal  experience  with  web  authoring.  The  reasons 
that  the  web  editor  (Macromedia  Dreamweaver)  was  not  highly  utilized  by  participants  to  develop  their  projects  can 
be  summarized  as  follows:  (a)  putting  projects  on  the  web  was  not  a requirement  for  this  course,  teachers  also  have 
the  option  to  submit  their  paper  in  a format  other  than  web  pages;  (b)  a steep  learning  curve  of  the  web  editor  might 
have  discouraged  beginners  from  continuing  using  the  program;  (c)  busy  work  schedules  kept  the  participants  from 
spending  time  in  learning  more  about  or  effectively  using  the  web  authoring  program  which  was  an  add-on  task  to 
project  development;  (d)  a lack  of  access  to  good  computers  might  have  deterred  teachers  who  had  no  experience 
with  computers  and  living  in  rural  areas. 

In  addition,  most  projects  developed  by  the  participants  did  not  involve  students  in  the  use  of  computer 
technology  in  their  learning.  In  spite  of  the  minimal  use  of  web  technology  in  developing  curriculum  by  most  of  the 
participants,  their  projects  did  employ  modern  environmental  technology  and  address  the  science-standards 
developed  by  DOE  at  the  State  of  Hawaii.  All  the  projects  showed  real-world  applications  and  participants  reported 
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successful  results  with  their  students.  All  lesson  plans  can  be  viewed  at  the  Malama  Web  site: 
http://hawaii.edu/malama/schools.html. 


Analysis  on  technology  use 

Overall,  email  and  the  Malama  W eb  site  were  the  main  means  of  communication.  In  the  Malama  listserv, 
the  majority  of  postings  were  submitted  by  the  instructors,  tech  support,  and  the  same  two  or  three  participants.  The 
majority  were  passive  users  of  the  communication  technology.  Technology  integration  was  most  successful  for 
projects  by  teachers  who  had  already  had  some  knowledge  in  using  the  web.  With  all  the  technological  support  and 
effort  to  make  technology  available  for  teachers,  most  teachers  would  only  use  technology  for  communication 
purpose,  not  for  project  development.  The  use  of  technology  was  a recommendation,  not  a requirement,  in  the 
course  syllabus  and  thus  partially  explains  the  lack  of  incentives  by  some  participants  in  the  incorporation  of 
technology  into  curriculum. 

The  constructivist  approach  to  encourage  context  learning,  knowledge  construction,  collaboration,  and 
conversation  would  have  worked  well  with  in-service  teachers  whose  skill  levels  are  more  or  less  at  the  same  level 
in  well-equipped  K-12  teaching  environments.  For  the  course  in  this  study,  the  participants  were  geographically 
dispersed  throughout  the  Hawaiian  islands  with  different  skill  levels  and  unequal  access  to  computers,  it  would  take 
more  than  one  technical  staff  to  provide  the  kinds  of  support  that  the  teachers  would  have  needed.  Furthermore, 
more  structured  tasks  in  motivating  teachers  to  use  technology  should  be  an  integral  part  of  the  course  design,  not  an 
add-on  task. 


Conclusion 

In  the  process  of  employing  technology  to  support  curriculum  development,  the  following  four  challenges 
in  most  rural  schools  need  to  be  addressed  first:  lack  of  computer  access,  computer  experience,  institutional  support, 
and  technical  support.  There  is  a need  to  provide  more  access  to  computers  and  technology  support  for  in-service 
teachers  before  any  technology  projects  can  be  implemented.  This  study  presents  a typical  scenario  in  teacher 
education  and  ways  to  provide  the  best  support  with  limited  resources.  It  is  not  a surprise  to  find  out  that  those 
teachers  who  are  successful  in  utilizing  technology  are  those  who  have  access  to  the  resources  in  this  study. 
Providing  computer  access  to  all  K-12  teachers  and  continuing  professional  development  should  provide  the 
foundation  toward  a successful  integration  of  technology  into  curriculum. 
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Abstract 

Over  the  past  three  years  or  more,  the  authors  have  been  examining  the  impact  of  technology  interventions 
on  students  and  teachers  - from  cultures  where  English  is  a second  language.  It  is  their  hypothesis  that  students 
and  teachers  from  non-English  speaking  nations  suffer  severe  disadvantages  when  technological  interventions  are 
superimposed  on  existing  traditional”  pedagogical  models.  This  paper  is  one  in  a series  that  examines  extent 
pedagogies,  primarily  in  Asia,  and  the  pressures  of  having  to  re-conceptualize  what  has  been  tradition  for  centuries. 
It  traces  significant  educational  origins  to  their  cultural  roots  and  examines  contemporary  initiatives  that  may  be 
disruptive.  It  offers  possible  solutions  that,  if  approached  with  reconciliation  as  its  focus,  might  offset  catastrophic 
results  and  achieve  pedagogical  symbiosis. 


Introduction 

Learn  as  if  you  were  following  someone  you  could  not  catch  up  to,  as  though  it  were  someone  you  were 
frightened  of  losing  (N.D.,  Confucius,  quoted  by  Beck,  2002). 

Thirteen  years  ago,  the  last  author  climbed  a mountain  named  Alishan  on  a cold  December  morning,  to  see 
the  sunrise  over  Central  Taiwan.  This  mountain  holds  special  significance  for  all  who  make  the  journey  and 
experience  the  sunrise,  for  it  is  both  a physical  and  a spiritual  renewal  of  one’s  inner  self. 

In  Islamic  lore,  Mohammad  went  to  his  mountain;  natives  and  international  tourists  alike  wait  for  the 
clouds  to  part  over  Mount  Kenya  to  see  its  rugged  peak;  and  innumerable  adventurers  aim  their  sights  on  the 
Matterhorn  (Mount  Cervin).  In  each  case,  there  are  those  like  we,  who  sought  renewal  and  understanding. 

The  mountains  we  describe  are,  barring  any  major  cataclysmic  event,  not  likely  to  move,  much.  From  this 
we  draw  our  own  parable  to  other  events  that,  while  not  as  ancient  as  the  mountains,  merit  connection  to  our 
analogy.  We  refer  to  the  plight  of  an  international  student  coming  from  his  or  her  country  to  another  for  study,  and 
finding  that  nothing  is  as  it  should  be. 

The  background  Asian  students  bring  to  the  West  reflects  an  educational  culture  that  saw  its  birth 
thousands  of  years  ago,  well  before  European  and  American  scholars  developed  their  educational  theories.  Plato 
(428-348  B.C.)  and  Socrates  (470-399  B.C.)  come  closest  as  peers  to  Confucius  (551-479  B.C)  (Beck,  2002)  for  it  is 
he,  and  his  contemporaries  who  shaped  the  way  education  is  currently  practiced  by  well  over  one  billion  people.  Or 
put  another  way,  over  a third  of  the  world’ s total  population.  There  are  myriads  of  lessons  to  be  learned  from  a 
system  that  has  continued  and  flourished  for  thousands  of  years.  What’s  so  good  about  it? 

This  article  is  about  the  impact  of  Confucian  pedagogy  on  thousands  of  Asian  students;  primarily  students 
from  China,  Hong  Kong,  Japan,  Korea,  Singapore,  and  Taiwan.  Despite  having  learned  most  of  their  lessons  in 
another  language,  English  has  now  become  the  “lingua  franca’’  for  students  from  across  all  Asia.  It  traces  the 
pedagogical  roots  of  a specific  group  of  international  students  (most  of  whom  are  from  either  China  or  Taiwan)  who 
have  come  to  one  university,  the  University  of  Central  Elorida  in  Orlando,  and  have  enrolled  in  one  graduate 
program,  the  Master’s  degree  in  Instructional  Systems  (http://pegasus.cc. ucf.edu/~instsys/,2002). 
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Its  premise  is  that  1).  Culture  does  make  a difference,  especially  when  two  cultures  have  the  possibility  of 
engagement  and  that  2).  These  cultures  are  generally  180  degrees  opposite  to  one  another,  sometimes  while  on  a 
near-collision  course. 

The  Problem 

Instructional  Systems  graduates  are  expected  to  develop  competencies  that  allow  them  to  communicate  to  a 
variety  of  audiences,  both  verbally  and  in  writing.  They  must  be  able  to  deliver  presentations  using  a wide  array  of 
technologies.  They  must  write  papers  that  reflect  both  eloquence  and  conviction.  They  must  work  well  as  a 
member  of  a multi-cultural  working  team  and  they  must  develop  positive  assertiveness.  The  competencies  expected 
of  them  are  listed  in  the  introductory  Instructional  Systems  course.  Survey  of  the  Application  of  Instructional 
Systems  (Cornell,  2001)  and  reiterated  in  Piskurich  & Sanders  (1998). 

Many  Western  students,  particularly  those  in  North  America,  are  generally  adept  at  using  these  skills,  given 
their  innate  tendency  toward  achievement  (judged  on  what  one  has  accomplished  and  one’s  past  record),  both 
professionally  and  socially.  Other  cultures  function  more  on  ascription  (status  attributed  to  one  by  birth,  kinship, 
gender,  age,  one’s  connections,  and  educational  record),  a phenomena  often  seen  in  Southern  Europe,  Latin 
America,  and  Asia  (Tropenaars  & Hampden-Turner,  1998,  p.  9).  For  Asian  students,  acting  on  many  of  the 
competencies  found  within  the  Instructional  Systems  program  is,  to  say  the  least,  unnatural,  uncomfortable,  and 
often  induces  considerable  stress.  The  result  — silence,  both  orally  and  in  written  documentation.  The  problems 
resulting  from  this  silence  are  in  two  categories.  Western  and  Asian: 

For  purposes  of  this  research,  when  we  use  the  word  “Asian,”  we  refer  to  students  who  came  to  UCF  from 
either  China  or  Taiwan.  Thus  far  we  have  no  students  from  either  Macao  or  Hong  Kong  but  recruitment  efforts  in 
those  areas  continue.  We  use  the  same  term  when  referring  to  teachers’  ethnicity.  The  overarching  descriptor  that 
guides  our  approach,  we  take  from  two  works  of  Trompenaars  & Hampden-Turner  (1997,  1998),  “reconciliation!” 

It  implies  neither  being  dominated  by  others  nor  acquiescing  but  rather,  “. . .through  an  integrative  process,  a 
universalism  that  learns  its  limitations  from  particular  instances,  for  example,  and  by  the  individual  voluntarily 
addressing  the  needs  of  the  larger  group  (p.  53,  1998).” 

In  this  instance,  the  “larger  group”  consists  of  the  Asian  students’  Western  peers,  who  number  a little  over 
half.  It  is  conceivable  that,  in  the  coming  years,  this  ratio  of  Asian-to-Western  students  may  well  change,  with  the 
numerical  balance  favoring  Asian  students!  Then  what?  The  larger  group’s  needs  were  not  immediately  apparent  as 
infusion  of  Asian  students  into  the  Instructional  Systems  master’s  program  was  not  a one-shot  event  but  gradual. 

We  recruited  one  student  at  a time,  until  the  critical  mass  gradually  increased  to  where  it  is  now  at  almost  half  the 
number  of  new  students  in  the  program.  A similar  trend  is  already  apparent  in  the  doctoral  program  in  Instructional 
Technology  where  the  critical  mass  of  Asian  students  exceeds  50% ! 

So... here  we  have  all  these  students  from  Asia  (as  well  as  those  from  Cyprus,  India,  Mexico,  Russia,  and 
Turkey)  and  each,  regardless  of  from  where  they  have  come,  brings  with  them  their  own  idiosyncratic  view  of  life. 

It  is  a pedagogical  life  that,  heretofore,  has  been  mostly  unknown,  by  both  the  students  and  their  Western  professors 
(2002,  International  Institute).  Many  professors  know  little  about  the  basics  involved  in  coming  to  the  United  States 
as  an  international  student;  they  know  even  less  as  to  the  specific  logistics;  and  even  less  than  less  in-depth 
knowledge  about  the  countries  from  which  their  international  students  have  arrived! 

Add  to  this  scenario  an  even  less  enlightened  American  student  population,  both  undergraduate  and 
graduate.  Our  North  American  students  are  mystified  by  the  silence  of  their  Asian  peers.  They  look  to  professors  to 
encourage  their  Asian  peers  to  communicate  because  they,  the  students,  have  little  success,  at  least  initially.  Many 
professors  remain  perplexed  for  they  also  find  their  Asian  students  to  be  mute.  How  to  evaluate  muteness  - that 
poses  their  immediate  challenge! 

The  professors,  and  often  the  students,  silently  ask  themselves,  “How  did  these  students  ever  get  accepted 
to  UCF?  They  can  barely  speak  English!”  A disablement?  Yes,  for  awhile,  but  not  a permanent  wound  so. . .the  task 
is  to  release  the  sounds  from  within  our  Asian  students,  to  hear  them  speak,  to  read  their  written  words,  and 
ultimately  to  sense  their  discovery  of  new  ideas,  just  as  these  same  professors  are  accustomed  to  doing  with  their 
North  American  students.  But  how  to  do  this. ..how  to  understand  the  pedagogical  culture  that  they  bring  with  them 
and  the  degree  of  symbiosis  between  that  culture  and  the  one  found  at  UCF  - that’s  the  challenge  as  well!  Yet,  we 
insist  that  these  students  learn  and  study  “our  way.” 

Our  research  is  an  attempt  to  understand  what  is  happening  with  our  Asian  students  so  we  can  reduce  the 
time  lag  between  the  time  a student  arrives  at  UCF  and  the  time  they  begin  to  truly  feel  comfortable  in  their  Western 
classroom  environments  - a task  that  is  formidable  and  complex.  Our  approach  is  to  seek  understanding  of  what 
these  students  bring  with  them  and,  to  do  so,  we  are  “going  to  the  experts,”  mainly  professors  from  both  Taiwan  and 
China,  asking  them  about  their  formative  education  years.  We  asked  them  to  describe  how  learning  was  for  them 
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when  they  were  students,  in  grades  anywhere  from  within  their  earliest  memory  of  schooling  to  the  time  they 
completed  their  last  degree.  In  this  way,  we  have  first-hand  descriptions  that  may  or  may  not  validate  what  appears 
in  the  literature. 

We  do  not  reject  what  the  literature  says  about  Asian  education,  but  we  wanted  to  add  richness  and  provide 
increased  depth  to  what  was  experienced,  thus  our  approach  was  and  is  to  conduct  live  video  interviews  where 
possible.  In  this  way  we  hope  to  validate  major  premises  found  in  the  literature  and  provide  a qualitative  database 
that  clearly  presents  imagery  that  communicates  the  contrasts  in  pedagogical  styles , East  and  West.  Note,  we  did 
not  say  East  vs.  West;  at  least  not  yet. 


Review  of  the  literature 

Even  a cursory  review  of  the  literature  suggests  images  that  stereotype  our  students,  be  they  from  China, 
Taiwan  or  North  America.  It  is  tempting  to  label  our  Asian  students  as  being  passive,  quiet,  silent,  submissive, 
respectful,  intelligent,  hard  working,  introspective,  disciplined,  and  many  other  descriptors  and,  for  some,  these  are 
accurate  in  their  assessment  (Tu,  2001).  Hampden-Tumer  and  Trompenaars  (1997,  op  cit.,  p.  10)  identify  values 
differences  between  the  West  and  East-Asia  in  the  business  sense  as  having  differing  historical  roots. 


West 

Supernatural  religion 
Belief  and  faith 
Cartesian  dualism 
Values  as  things 
Cultures  and  Values -Yin 
Pioneer  capitalism 
Einite  Games 


East 

Secular  humanism  and  enlightenment 
Paradigmatic  assumptions 
The  Way  of  Complimentarity 
Values  as  wave-forms 
Cultures  and  Values  - Yang 
Catch-up  capitalism 
Infinite  Games 


We  include  these  values  sets  because  there  is  a continuing  relationship  between  instructional  systems 
designer  competencies  expected  in  the  West  by  business  and  industry  and  the  extent  conditions  the  Asian  students 
bring  with  them.  This  relationship  often  reflects  the  Yin  and  Yang  depicted  above.  Hampden-Turner  and 
Trompenaars  define  belief  differences,  those  in  the  West  being  anchored  in  “...the  commandments  of  supernatural 
beings  and  their  sanctions  for  good  or  bad  behavior  in  an  after-life”,  (p.  10- 11).  In  East  Asia  the  authors  cite  value 
systems  that,  while  varied  across  different  countries,  are  based  on  “their  wisdom  being  secular,  practical  and  of  this 
world.”  (p.  1 1).  That  “they  are  ‘humanistic’  in  the  sense  that  they  aspire  to  improve  the  human  and  social 
condition,  and  failure  in  this  leads  to  criticism  and  reappraisal,  (p.  11). 

It  is  tempting  to  label  our  North  American  students  as  being  Type  A,  competitive,  brash,  aggressive,  loud, 
spontaneous,  wild,  lacking  in  respect,  superficial,  ego-centric,  lazy,  having  little  discipline  or  motivation,  etc.  and  for 
some,  these  too,  are  accurate  assessments.  Let’s  remove  the  silk  gloves  and  tell  it  like  it  is  - there  are  students  from 
China  and  Taiwan  that  could  easily  pass  for  what  we  have  described  as  being  stereotypes  of  North  American 
students  and...  there  are  North  American  students  who  emulate  many  of  the  attributes  seen  in  the  literature  as  being 
typical  reflections  of  Asian  behavior  and  values. 

In  the  literature  we  found  ample  evidence  of  how  things  are.  We  found  little  of  help  as  to  what  to  do  about 
it,  especially  as  relates  to  ways  in  which  we  might  make  the  lives  of  the  Chinese  students  in  our  program  more 
positive.  We  identified  a number  of  categories  wherein  insight  was  provided  as  to  what  beliefs  the  students  bring 
with  them.  As  Trompenaars  and  Hampden-Turner  assert,  however,  “you  can  never  understand  other  cultures  (1998, 
p.  1)  and  so,  while  it  is  our  intent  to  provide  clarity  as  to  Asian  thinking,  understanding  might  lie  beyond  our  real 
capabilities.  The  best  we  can  hope  for  is  to  create  sensitivity  toward  the  problems  raised,  and  in  doing  so,  perhaps 
emerge  with  observations  and  strategies  that  may  make  life  for  our  students  far  better. 

In  support  of  this  caveat  is  a statement  read  from  another  source,  now  disremembered  {Anonymous,  N.D.), 
that  described  the  feelings  of  an  American  expatriate  in  China  who  had  worked  for  20+  years  there.  He  was  asked  : 
“With  all  your  years  living  in  China,  how  long  did  it  take  you  to  understand  the  Chinese?”  His  replied,  “I  still  do 
not  understand  them  and  probably  never  will.”  He  added:  “One  cannot  understand  those  from  another  culture  unless 
they  have  been  born  or  grown  up  with  it!”  So  much  for  really  understanding  our  Chinese  students  in  Orlando. 

In  terms  of  what  is  valued  and  how  it  is  manifested  within  the  UCE  classroom,  the  literature  reinforces  our 
own  experiences.  An  ongoing  concern  is  the  silence  of  Chinese  students  in  our  classes,  especially  during  their  first 
or  second  semesters  with  us.  Is  silence  always  “bad?”  This  question  was  posed  us  at  an  Orlando  conference  of  the 
Comparative  and  International  Education  Society  (CIES)  by  Dr.  Haiyan  Hua,  Senior  Research  Associate  of  Harvard 
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University’s  International  Education  Group.  We  were  quick  to  respond  in  the  negative,  that  no,  there  are  many 
times  when  silence  is,  in  fact,  “golden.”  From  this  exchange  we  learned  that  we  needed  to  affirm  our  target 
population  as  being  Chinese  students  enrolled  in  a specific  academic  graduate  program  that,  while  valuing  silence  in 
many  activities,  it  also  demands  vocal  and  written  participation  in  many  others. 

We  found  evidence  of  why  our  Chinese  students  were  silent  in  Brooks  (1997)  who  was  quoting  Yum 
(1994):  ‘...the  Chinese  communication  process  places  the  ‘emphasis  ...on  the  receiver  and  listening  rather  than  the 
sender  or  speech  making’  “(p.83).  Brooks  emphasizes  Yum’s  point  that  Chinese  are  receivers  of  messages,  listening, 
rather  than  being  senders.  She  provides  a pedagogical  blueprint  as  to  the  nature  of  how  Chinese  students  are 
educated: 

Confucius  believed  that. . .a  hierarchal  system  was  essential  to  the  harmonious  well  being  of  society.  This, 
in  turn,  is  reflected  in  the  Chinese  classroom.  Chinese  students  regard  their  teacher  as  all  knowing,  and  the 
absolute  authority  on  the  subject  matter.  Due  to  the  rigid  teacher-student  relationship,  Chinese  teachers  are 
under  severe  pressures  not  to  make  mistakes,  not  to  misguide  students,  and  not  to  be  criticized,  in  order  to 
maintain  their  “all  knowing”  and  “ever  correct”  status.  It  is  the  duty  of  students  to  give  utmost  respect  to 
the  teacher.  To  ask  questions  of  the  teacher,  or  to  question  the  words  of  a teacher  is  tantamount  to 
questioning  the  position  of  the  teacher,  and  therefore  is  not  a feature  of  Chinese  classrooms.  Since  the 
teacher  is  the  sole  authority  in  the  classroom,  rigid  order  and  formality  are  the  main  features  of  the  Chinese 
learning  environment  (Su,  1995). 

Zhang,  Sillitoe  and  Webb  (1999),  in  quoting  Ballard  & Clanchy  (1991)  compare  Western  and  Chinese 
differences  in  the  attitudes  toward  learning: 

“...in  Western  culture,  tertiary  education  is  oriented  towards  extending  knowledge.  Therefore,  the  teaching 
approach  used  and  learning  approach  encouraged  are  designed  to  develop  analytical  and  speculative 
abilities  of  students.  By  comparison,  in  more  ‘traditional’  cultures,  for  example  Chinese  culture,  the 
education  systems  are  mainly  oriented  towards  conserving  knowledge,  and  the  learning  approach  fostered 
emphasizes  the  reproductive  ability  of  students. 

Tu  (1999)  reinforces  these  findings  by  stating  that  Chinese  teachers  have  been  accustomed  to  teaching 
where  there  was  only  one-way  communication  and  a quiet  environment.  This  no  longer  exists.  All  subjects  are 
discussed,  even  the  teacher’s  private  life.  Teachers  fear  the  loss  of  their  authority  and  the  pressure  of  public  opinion 
(p.  3). 

In  a later  work,  (Tu,  2001),  he  adds  that  Chinese  collect  information  from  non-verbal  channels  and  perceive 
more  exact  information  than  has  been  delivered  (p.  5).  Lin  and  Yi  (1997)  add  more  pieces  to  our  puzzle  when  they 
describe  further  issues  faced  by  Chinese  students  that  compliment  those  already  discussed: 

International  students  from  Asian  countries  are  often  stereotyped  as  quiet,  reserved  and  non-assertive. 

These  cultures  place  an  emphasis  on  harmony  and  respect  for  authorities.  Therefore,  many  of  them  are 
reluctant  to  share  their  feelings  or  emotions,  express  their  opinions  or  oppositions  to  anyone,  especially  to 
authority  figures.  Thus  instead  of  emphasizing  personal  rights  and  assertive  communication,  Asians  tend  to 
emphasize  the  importance  of  patience,  harmony,  respect  and  deference.  Asian  cultures  tend  to  place  a high 
value  on  team  efforts  of  collectivity  whereas  Western  culture  tends  to  emphasize  individualism.  Asians  are 
also  modest  about  their  accomplishments.  Many  Asian  international  students  feel  uncomfortable  with  the 
individualism  and  the  competitiveness  associated  with  the  American  culture. 

This  is  not  to  suggest  a caveat,  a series  of  excuses  to  fall  back  upon,  rather,  it  is  a reflection  of  reality  for 
many.  We  cannot  stereotype  our  students  so  easily,  be  they  from  Asia  or  Central  Florida.  The  common 
denominator,  metaphorical  in  many  respects,  seems  to  be  what  was  termed  the  “figure -ground”  relationship 
(McLuhan  1964),  i.e.,  that  students  will  perform  within  parameters  that  are  related  to  their  most  comfortable 
environment,  whatever  suits  them  at  the  time. 

In  “McLuhanese”  this  concept  is  expressed  as:  'Figure'  refers  to  something  that  jumps  out  at  us,  something 
that  grabs  our  attention.  'Ground'  refers  to  something  that  supports  or  contextualizes  a situation,  and  is  usually  an 
area  of  Mnattention  (Gow,  1998). 

Mercurial  in  many  ways,  chameleon-like  in  adaptability  - it’s  just  what  students  do  to  survive!  The 
literature  excerpts  provided  are  but  a tip  of  the  pedagogical  iceberg.  It  is  our  intent  to  continue  surveying  the 
literature  continually  to  better  gain  understanding  of  our  own  Chinese  students. 

We  are  in  the  process  of  developing  methodology  that  will  enable  us  to  better  understand  the  implications 
implicit  in  teaching  Chinese  students  at  UCF.  We  think  that  our  findings  may  also  prove  useful  to  other  North 
American  institutions  wherein  Instructional  Technology  programs  are  offered.  Ultimately,  we  envision  the  results 
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as  also  being  of  considerable  use  to  multinational  corporations,  many  of  whom  hire  many  Asian  employees.  An 
initial  description  of  our  methodology  follows. 

Be  advised  that  what  we  include  in  this  article,  as  well  as  in  our  presentations,  may  appear  offensive  to 
some,  to  marginalize  some,  to  be  found  guilty  of  being  patronizing  to  others. . .none  of  which  is  intended.  We  are 
taking  observations  as  we  get  them,  subjecting  them  to  analysis,  comparing  them  to  other  research  found  in  the 
literature,  and  then  attempting  to  devise  useful  strategies  that  will  enable  our  Chinese  and  Taiwanese  students  to  not 
only  succeed,  but  to  prosper!  In  effect,  we  are  trying  to  level  the  academic  playing  field  at  the  University  of  Central 
Florida  so  that  there  is  equity  for  most,  if  not  all. 


What  we  were  trying  to  do 

The  genesis  of  this  study  began  a few  years  ago  after  the  lead  author,  having  recruited  numerous  students 
from  Taiwan  and  China  into  the  Instructional  Systems  master’s  program  at  UCF,  found  that,  while  each  student 
brimmed  with  enthusiasm,  when  it  came  time  for  communicating  ideas  in  class,  most  remained  silent.  The  students 
were  always  pleasant,  respectful  beyond  belief,  and  had  a work  ethic  akin  to  our  forefathers.  In  class,  when  asked  a 
question  directly  (as  is  typical  in  North  American  classrooms),  they  would  look  down  at  their  notes,  mumble  soft 
words,  most  of  which  were  unintelligible,  and  squirm  uncomfortably  in  their  seats,  knowing  that  multiple  eyes  were 
upon  them.  When  asked  to  post  messages  on  an  online  discussion  forum,  the  most  one  could  anticipate  from  our 
Asian  students  was  a perfunctory  few  words  of  acknowledgement,  “Yes,  I agree  with  her  statement’’  or  similar 
responses.  Not  much  to  go  on  as  an  evaluative  tool.  Then  there  were  the  infamous  synchronous  chat-rooms. . .where 
written  paralysis  was  the  rule  for  our  Asian  students,  not  the  exception.  The  odds  of  the  professor  really  knowing 
what  the  student  knew  were  getting  slimmer  by  the  moment. 

Another  caveat:  The  instructor  in  this  instance,  was  one  who  evaluated  performance  based  on  what  the 
students  do,  tied  to  performance  objectives  and  goals  that,  more  often  than  not,  include  project-based  assignments, 
rather  than  conventional  paper  and  pencil  objective  tests.  In  addition  to  individual  and  group  assignments,  emphasis 
is  placed  on  participation,  on  the  development  and  continuation  of  positive  attitudes,  of  being  self-motivated  rather 
than  having  to  be  told  what  page  to  read  or  how  many  words  were  necessary  in  a paper  to  receive  an  A.  He  placed 
continual  emphasis  on  teamwork,  oral  and  written  communication,  as  opposed  to  memorization  of  facts  and  figures. 
In  short,  his  pedagogical  model  was  180  degrees  opposite  that  typically  followed  by  his  Asian  students! 

“Well,’’  one  asks,  “why  is  such  communication  so  necessary?’’  Answer?  Because  the  Instructional 
Systems  master’s  program  (and  subsequent  lead-in  to  a Ph.D.)  at  UCF  has  both  written  and  verbal  communication  as 
foundation  skills  as  well  as  development  of  assertiveness,  teamwork,  and  numerous  additional  competencies.  Taken 
collectively,  this  skills  menu  posed  a serious  threat  to  the  previous  learning  models  brought  us  by  our  students  from 
China  and  Taiwan.  Asian  culture  does  not  particularly  reward  self-efficacy,  assertiveness,  or  the  typical  North 
American  capability  of  being  at  ease  speaking  in  front  of  one’s  peers.  The  Western  notion  of  one  being  a “hale 
fellow,  well  met’’  is  an  alien  concept  to  the  Asian  mind. . .business  is  not  done  via  bravado  or  exuberance,  rather,  by 
quiet  and  deliberate  analysis  of  the  facts,  followed  by  determined  action  of  benefit  to  all,  not  to  the  individual.  With 
the  foregoing  as  basis,  what  methodology  could  we  find  that  might  begin  to  shed  light  on  our  questions? 

Interviewee  Questions 

The  researchers  asked  two  basic  questions  of  the  Chinese  and  Taiwanese  subjects.  The  subjects  in  three 
groups  were  teachers  or  professors  from  China  and  Taiwan.  The  fourth  group  was  comprised  of  UCF  students 
enrolled  in  the  Instructional  Systems/Instructional  Technology  graduate  program  areas. 

The  questions  were: 

1.  When  you  were  going  to  school  (meaning  any  time  between  your  primary  school  and  graduate 
education),  how  were  you  taught  and  what  were  the  recurring  values  that  your  teachers  instilled  in 
you? 

2.  Now  that  you  are  a teacher/professor,  what  strategies  do  you  use  when  you  teach  and  what  values  do 
you  feel  are  important  to  instill  in  your  students? 

3.  A fourth  group  of  both  Chinese  and  Taiwanese  was  composed  of  current  students  enrolled  at  the 
University  of  Central  Florida.  In  addition  to  question  I,  above,  they  were  asked,  “How  are  the 
instruction  and  the  values  you  are  learning  here  at  UCF  different  than  that  you  received  at  home?’’ 
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Methodology 

Interviews  are  being  conducted  with  four  separate  and  distinct  groups  of  individuals,  both  composed  of 
either  teachers/professors  or  students  from  China  and  Taiwan.  By  using  these  individuals  as  SME’s  (subject  matter 
experts),  we  were  authentic  in  our  selection  of  subject  class.  We  then  needed  to  differentiate  between  the  groups  to 
provide  focus  on  differences  that  might  exist  between  them.  We  decided  to  use  four  groups  of  ten,  an  N of  40.  The 
groups  consisted  of  the  following: 

1.  Teachers/professors  who  had  been  educated  in  Taiwan  or  China  and  had  not  studied  abroad.  We 
anticipated  that  the  demographics  of  this  group  might  be  those  who  were  more  experienced  and  whose 
teaching  would  mirror  much  of  the  methodology  they  learned  from  their  own  teachers. 

2.  Teachers/professors  educated  in  Taiwan  or  China  who  had  completed  additional  study  in  the  West,  be 
it  in  North  America,  Europe,  Australia,  New  Zealand,  etc.  and  had  since  returned  to  teach  in  their 
homeland. 

3.  Professors  currently  teaching  at  the  University  of  Central  Florida  from  China  or  Taiwan. 

4.  Students  from  China  or  Taiwan  studying  at  the  University  of  Central  Florida,  preferably  those  enrolled 
in  graduate  Instructional  Technology  programs. 

Previously  we  described  the  questions  that  would  be  asked  of  those  in  each  group.  What  we  were  (and  are) 
trying  to  do  is  to  get  a clear  picture  of  how  our  Chinese  and  Taiwanese  students  learned  and  what  values  they  bring 
with  them  when  they  come  to  UCF. 

Each  subject  is  asked  the  questions  while  being  recorded  on  video.  The  sessions  are  relatively  brief,  given 
that  there  are  only  two  basic  questions  being  asked.  The  subjects  may,  if  they  wish,  reply  in  either  Chinese 
(Mandarin)  or  English.  If  Mandarin  is  selected,  the  interviewer  speaks  in  Mandarin  as  well.  Transcripts  of  each 
interview  are  produced  and  analyzed.  Where  the  interviews  are  in  Mandarin,  closed-captioned  English  will  be 
inputted  on  those  taped  interviews  and  an  English  narration  provided.  An  analysis  will  then  be  made  of  key  words 
across  all  groups  and  recorded  in  terms  of  similarity,  frequency  of  use,  and  by  category  of  interviewee.  A matrix 
will  then  be  constructed  that  depicts  the  relationships  between  key  words  used  and  group  type,  thus  profiling  group 
differences  in  a more  visible  manner.  An  edited  tape  of  relevant  excerpts  will  be  abstracted  from  the  raw  footage 
that  highlights  the  results  of  the  taped  interviews. 

Once  the  analysis  of  Asian  (Confucian)  pedagogical  methods  has  been  validated,  efforts  will  be  made  to 
affect  a comparison  between  these  methodologies  and  those  regularly  employed  within  Western  university 
classrooms,  more  specifically,  those  wherein  Instructional  Systems  design  students  are  enrolled. 

Research  Questions 

Are  there  elements  of  the  valued  Confucian  pedagogy  that  might  be  compatible  with  those  employed  in  the 
West?  Are  there  elements  of  Western  pedagogy  that  might  be  in  harmony  with  Confucian  pedagogies?  Are  there 
implications  that  this  research  might  have  for  students  from  other  cultures?  Are  the  findings  of  value  to  technology- 
based  commercial  organizations  who  engage  in  multinational  commerce,  especially  given  the  current  and  projected 
increase  in  such  initiatives  in  East  Asia?  Is  there  symbiosis  or  asymbiosis  - that  is  the  goal  we  seek  to  address? 

Preliminary  Results 

Over  the  past  three  years  a number  of  efforts  have  been  made  (Cornell,  Ku,  Lee,  Pan,  Tao  & Tsai,  2000; 
Cornell  & Elshennawy,  2001;  Cornell  & Tao,  2001)  to  develop  pedagogical  strategies  that  would  be  effective  in 
reaching  our  students  from  China  and  Taiwan.  These  included: 

1.  Giving  students  advance  notice  of  written  or  oral  reports  with  topics,  dues  dates,  and  expectations 
clearly  identified. 

2.  Encouraging  students  to  prepare  and  post  individual  Power  Point  slides  on  the  class  discussion  board 
that  introduced  them  to  their  North  American  peers  and  requiring  the  North  American  students  to  do 
likewise. 

3.  Providing  a “safe  harbor’’  for  the  first  semester  to  allow  all  students  increased  latitude  in  terms  of 
spelling,  grammar,  and  punctuation  during  postings,  especially  while  engaged  in  synchronous  chat 
sessions  and  in  class,  regarding  pronunciation,  when  oral  discussion  is  being  encouraged. 

4.  Encourage  (but  not  initially  requiring)  Asian  students  to  NOT  work  together  in  the  same  project 
groups,  thus  providing  the  Asian  students  with  direct  opportunities  and  challenges  in  using  English. 
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5.  Inviting  students  fro  m both  Asia  and  North  America  to  professional  meetings,  social  functions,  even  to 
the  professor’s  home,  to  provide  a “mutually  safe  harbor”  for  differing  cultures. 

6.  Development  and  production  of  a series  of  video-based  modules  related  to  assertiveness  (Cornell,  Pan, 
Rendon,  Hutton  & Sheehy,  2002a). 

7.  Development  of  courses  related  to  international  issues  and  the  application  of  instructional  technology 
(Cornell,  2001b,  2002b) 

We  hope  that  a matrix  of  the  data  collected  will  reveal  the  close  correlation  between  what  is  found  in  the 
literature  and  what  was  articulated  by  the  subjects  during  their  interviews.  A draft  of  what  the  matrices  may 
resemble  follows. 

Confucian  pedagogical  influences  upon  Chinese  and  Taiwanese  professors/teachers  and  their  students 

Question  1 : Think  back  to  when  you  were  a student,  to  any  time  between  your  primary  school  years  and  when  you 
obtained  your  higher  education  degree.  How  were  you  taught?  What  values  did  you  learn? 


Teacher  used 
blackboard 

Teacher  is 
“God” 

And 

someone 
for  whom  I 
had  great 
respect 

Afraid  of 
my 

teacher 

I had  to 
always  be 
quiet  and 
rarely  ever 
asked  a 
question 

I had  to 
listen  and 
take  notes 
of  what 
the 

teacher 

said 

Little  or  no 
interaction, 
unless  it  was  a 
science  type 
lab  class 

Teachers  used 
technology 

Group  1 

Group  2 

Group  3 

Group  4 

Group  1 - Professors  or  Teachers  with  no  overseas  teaching  experience  or  degrees 

Group  2 - Professors  or  Teachers  with  overseas  teaching  experience  and  one  or  more  degrees  received  abroad 

Group  3 - Chinese  or  Taiwanese  professors  teaching  at  the  University  of  Central  Florida 

Group  4 - Chinese  or  Taiwanese  Instructional  Systems  students  at  the  University  of  Central  Florida 

Note:  Question  1 is  modified  for  Group  4 to  read:  “Think  back  to  when  you  were  a student,  any  time 
between  primary  school  and  when  you  came  to  UCF.  How  were  you  taught  and  what  values  did  you  learn?” 

Both  the  matrices  being  developed  will  use  key  words  and/or  phrases  mentioned  during  the  interviews 
across  the  horizontal  axis  with  the  different  groups  aligned  along  the  vertical  axis. 

Question  2:  Now  that  you  are  a teacher,  how  do  you  teach  and  what  values  do  you  instill  in  your  students? 


I started 
out  being 
very  strict 

I continue 
to  be  very 
strict  with 
my 

students 

I am 

gradually 

changing 

my 

teaching  so 
it  includes 
more 

interactivity 

I have 

moved 

away  from 

being 

teacher 

centered  to 

being 

student 

centered 

I use  a lot 
of 

technology 
in  my 
classes 

While  I 
respect  my 
teacher  and 
his  or  her 
knowledge, 
I know  that 
there  are 
also  other 
sources  of 
information 

I continue 
to  teach  to 
the  test 

Group  1 

Group  2 

Group  3 

Group  4 

Ill 


Group  1 - Professors  or  Teachers  with  no  overseas  teaching  experience  or  degrees 

Group  2 - Professors  or  Teachers  with  overseas  teaching  experience  and  one  or  more  degrees  received  abroad 

Group  3 - Chinese  or  Taiwanese  professors  teaching  at  the  University  of  Central  Florida 

Group  4 - Chinese  or  Taiwanese  Instructional  Systems  students  at  the  University  of  Central  Florida 

Note:  This  question  was  modified  for  Group  4 to  read:  “Now  that  you  are  a student  at  UCF,  how  are  you 
taught  and  what  values  do  you  get  from  your  instructors?” 

Current  and  Anticipated  Results 

This  is  a longitudinal  study  in  that,  as  time  and  financial  resources  permit,  the  data  collection  will  continue 
until  there  is  a balance  in  intra -group  findings.  Thus  far,  we  have  obtained  two  interviews  in  China,  both  of  which 
are  from  Group  1.  It  is  hoped  that  in  November  2002  an  additional  six  or  eight  subjects  from  China  will  be 
interviewed,  three  of  whom  will  be  in  Group  1 with  the  remaining  five  to  be  in  Group  2.  We  have  interviewed  10 
from  Taiwan  in  Group  1,  and  three  from  Group  2.  The  remaining  two  from  Taiwan  in  Group  2 will  also  be 
interviewed  in  November  2000.  Groups  3 and  4 will  be  interviewed  during  December  2002. 

Following  is  a sampling  of  some  of  the  interviews  conducted  thus  far.  They  are  summary  statements 
arranged  according  to  categories  that  will  eventually  appear  in  the  matrices  being  developed. 

Data  Analysis  and  Results 

What  it  was  like  when  I was  a student  is  included  in  the  first  section  and  five  key  points  were  identified. 
They  are  educational  media,  deification  of  the  teacher,  one  way  interaction,  transformation  in  the  college,  and 
normative  influences. 

Educational  media 

Most  of  educational  media  employed  included  chalk  and  board,  pictures,  and  tape  recorders.  A Shanghai 
interviewee  from  Group  1 (who  has  no  degree  from  another  country)  stated: 

“In  the  50' s young  people  began  to  learn  English,  but  these  teachers  were  very  few.  So  I was  taught  by  lots 
of  old  teachers  from  a different  era.. .they  just  teach  by  the  book  and  prepared  lessons  by  themselves.  There 
were  very  few  tape  recorders.  We  listened  to  the  teacher  read  and  we  have  very  few  questions.  ” 

In  the  past,  educational  media  was  not  as  popular  as  it  is  now.  A faculty  member  with  an  Ed.D.  from  an 
American  university  added: 

“In  my  time,  they  didn  't  have  multimedia,  video;  the  only  thing  we  had  was  a Studio  Classroom... on  the  air. 
Students  could  listen  to  the  radio.  Now  the  students  in  Taiwan  are  very  lucky  in  that  they  have  lots  of 
professors  and  teachers  from  U.S.  with  majors  in  TEST.  We  had  chalk  and  board.  ” 

There  are  no  major  differences  found  between  Group  1 and  Group  2 in  terms  of  educational  media  used, 
which  is  not  to  our  surprise.  As  technology  evolves,  more  educational  media  are  anticipated  and  expected  to  be 
employed  in  the  organizations  and  institutions. 

Deification  of  the  teacher 

The  role  of  a teacher  was  deified  and  amplified  due  to  its  tradition  (i.e.,  Confucius  teachings).  Teachers 
were  treated  as  an  authority  in  knowledge,  values,  and  life  experiences.  A department  head  with  30  years  experience 
said: 

“/  went  to  a national  normal  university  in  Taiwan.  The  school  was  the  only  institution  at  that  time 
to  prepare  any  individual  for  being  an  effective  teacher.  During  my  studies  there,  I was  taught  to  be 
disciplined.  I was  not  allowed  to  be  a rascal.  I was  taught  about  professionalism.  I was  not  allowed  to  do 
what  students  of  other  schools  did.  It  was  close  to  being  military  training... When  I was  a student,  I thought 
teachers  were  strict  and  tough.  I felt  they  were  frightening,  even  more  than  my  parents...  ” 

Regardless  of  how  sufficient  their  knowledge  and  skills  may  have  been,  individuals  were  taught  by  parents 
to  be  obedient  and  humble  before  teachers  and  they  were  required  to  look  up  to  their  teachers.  Mr.  Lin,  an 
interviewee  who  is  an  education  commentator  and  had  resided  in  the  States,  recalled  what  he  had  experienced  as  a 
child: 

“What  I was  taught  when  I was  young  was  of  course  in  the  style  of  Confucius.  We  were  taught  all  the 
Confucian  values.  Like,  we  should  respect  our  teachers.  We  should  be  humble.  We  should  not  speak  too 
much.  Just  listen,  don 't  talk  too  much.  And  don 't  try  to  raise  too  many  questions;  because  some  time  we 
will  think  that  you  are  not  so  humble  to  your  teacher  and  we  will  train  you  in  such  a way  that  silence  is 
God.  Don 't  talk  too  much.  ” 
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As  a result,  teachers  were  among  the  top  of  the  social  hierarchy.  Their  teachings  were  like  truth.  Maybe 
that  is  one  of  the  reasons  why  Confucian  teaching  remains  one  of  the  classics  and  must-reads  for  the  educated. 

Interaction 

One-way  interaction  between  instructors  and  students  was  common.  The  instructors  lectured  and  students 
listened  carefully  and  bore  in  mind  what  was  told  them.  Each  of  the  two  parties  were  held  responsible  for  their  own 
tasks.  Dr.  Chien,  a professor  of  Social  Studies  Education  at  the  Taipei  Municipal  Teachers  College  in  Taipei,  who 
received  his  doctorate  at  Northern  Arizona  State  University,  mentioned: 

“To  be  a Chinese  student,  all  the  teachers  teach  the  students  directly,  just  use  oral  teaching  strategy.  There 
is  no  direct  response.  ” 

As  one  may  perceive,  lecturing  was  a common  strategy  employed  by  teachers  in  the  classroom.  Not  only 
were  students  asked  to  be  good  listeners  and  note-takers,  but  it  was  demanded  that  they  retain  the  information  as 
much  as  they  possibly  could.  Although  it  does  not  sound  interactive,  students  did  not  fight  against  the  authorities. 

Transformation  at  the  college  level 

Academic  freedom  in  the  college  was  a backlash  against  conventional  teaching.  More  interaction  was 
found  at  this  level.  Richard  Wang  commented  on  his  college  learning: 

When  I studied  in  the  university,  it  was  a free  environment.  We  needed  to  get  information  from  the  library; 
we  needed  to  share  experiences  with  others.  When  I entered  graduate  school,  microcomputers  became 
popular  in  Taiwan.  It  influenced  the  learning  method,  learning  environment,  and  teaching  method. 

Mr.  Huang,  another  interviewee  from  an  elementary  school  in  Taipei  echoed  Richard  Wang  and  said  that 
he  was  totally  transformed  in  the  college: 

“The  instructors  played  a role  of  knowledge  facilitator...  they  spared  more  time  for  communication  between 
them  and  students,  which  is  different  from  when  I was  in  high  school. . .regardless  of  their  high  social- 
economic  status.  ” 

His  instructors  encouraged  him  to  ask  questions  without  any  concern.  He  was  required  to  do  research  and 
present  in  front  of  the  class.  This  affected  him  as  a pre -service  teacher. 

Norm 

Not  only  parents  and  teachers  disciplined  students  and  minors,  but  also  society  has  an  had  an  impact, 
effecting  students’  behavior  and  cognition.  Dr.  Tsai  of  National  Chin  Hua  University  in  Taiwan  addressed  this 
issue: 

“When  we  were  little  children,  the  meaning  of  being  a good  student  is  to  work  hard,  to  behave.  That’s 
typically  the  Asian  students ' philosophy.  After  we  grow  up,  we  got  more  responsibility.  That  means  you 
have  to  do  good  in  your  studies  and  then  you  can  be  a useful  person.  Useful  probably  means  you  have  to  do 
good  things  to  not  only  yourself  but  others.  ” 

Once  environmental  factors  start  to  regulate  students’  behavior  and  way  to  learn,  the  influence  is  to  a great 
degree  on  habituation  with  respect  to  normative  influences.  Mr.  Lin,  an  educational  reformer,  would  like  to 
“change”  the  value  systems,  so  as  to  improve  the  way  to  teach  and  learn  for  teachers  and  students.  He  suggested: 

“/  tried  to  modify  the  value  system  to  my  students.  Those  things,  which  are  very  good,  very  helpful  or  very 
inspiring  teaching  methods,  I tried  to  encourage  the  students  to  ask  questions.  You  have  read  the  article  in 
“higher  education  curriculum” . I think  something  is  very  well... part  of  it.  Not  too  much.  Students  were 
already  molded  into  a certain  kind  of  style  of  education.  ” 

A huge  task  like  this  demands  a leader  of  strong  will  and  much  additional  support. 

What  it  is  like  now  I am  a teacher  follows  and  another  five  major  ideas  were  represented:  strictness  and 
discipline,  attitude  shift,  educational  technology,  sources  of  information,  and  measurement  and  evaluation. 

Strictness  and  discipline 

Strictness  and  discipline  are  major  concerns  to  some  teachers,  especially  those  in  K- 12  classrooms.  When 
technical  glitches  occurred,  they  had  to  resolve  the  problems  and  manage  the  classroom  at  the  same  time.  Students 
at  K- 12  levels  are  not  as  self-regulated  and  self-disciplined  as  those  at  higher  education.  The  K-12  classroom 
teachers  tended  to  shorten  their  distance  to  the  students;  however,  in  the  meantime,  they  would  like  to  maintain  their 
authoritarian  role,  lest  the  students  disrespect  them.  Mr.  Lee,  an  elementary  school  teacher,  claimed  that: 

“/  was  a harsh  teacher  when  I was  a cub  teacher.  I reflected  on  my  own  teaching  and  leading  role  over 
time.  1 will  not  allow  my  students  to  get  too  close  to  me,  because  they  are  still  kids  and  they  can  easily 
jump  on  your  head.  ” 
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Conversely,  another  elementary  school  teacher  stated  that  she  loved  her  students  as  her  children.  She 
always  included  her  daughter  as  her  target  audience  when  designing  teaching  plans.  For  college  professors,  these 
sentiments  are  not  necessarily  an  issue. 

Attitude  shift  (teacher-centered  to  student -centered) 

A teacher’s  attitude  toward  teaching  changes  over  time.  Number  of  years  spent  teaching  and  environmental 
factors  are  major  factors  in  this  change.  A school  principal  with  40  years  teaching  experiences  suggested: 

“Over  time  the  role  of  teachers  has  changed.  Environmental  factors  play  a critical  rule  in  this  issue. 
Students  are  affected,  too.  ” 

How  does  it  change  or  evolve?  An  elementary  school  teacher: 

“The  way  I teach  is  totally  different  from  what  my  teachers  taught.  It  has  been  from  teacher-centered  to 
student-centered.  ” 

Paradigm  shifts  result  in  teacher  attitude  shifts.  Based  on  the  interviews,  most  teachers  appeared  passive  in 
dealing  with  these  paradigm  shifts. 

Educational  technology 

New  technologies  are  increasingly  and  continually  being  introduced.  Thanks  to  these  innovations,  students 
are  provided  with  alternative  learning  experiences.  Even  though  the  relationship  of  technology  to  learning  has  been 
questioned,  technology  plays  a critical  role  in  determining  learning  and  motivation.  Mr.  Xie  of  Beijing  asserted: 

“We  now  use  audio  video  material  for  teaching.  We  use  audio  tapes  video  tape  VCD,  DVD,  and  other 
software  to  teach  the  students.  So  students  can  learn  by  themselves  at  home  or  in  the  classroom,  and  we  use 
the  platform  of  computer  to  provide  information  or  knowledge  about  the  computer.  If  the  students  want  to 
learn,  they  can  learn  by  themselves  anytime.  Also  in  the  classroom  we  asked  students  to  ask  questions.  In 
this  way,  we  have  two-way  communications.  Before  we  only  have  one-way  communication.  ” 

Self-paced  learning  is  now  receiving  more  acceptance.  Teacher  attitudes  toward  technology  use  may 
contribute  to  its  popularity.  A caveat  is  proposed  by  Richard  Wang,  a faculty  of  National  Taiwan  Normal 
University: 

“It  is  a new  technology  environment  including  multimedia  and  the  Internet.  I like  to  teach  my  students  and 
discuss  with  my  students  on  the  Internet.  Teach  effectively  and  learn  well  and  happily.  In  attaining  so, 
advanced  technologies  are  not  really  necessary  at  all  times.  ” 

Sources  of  Information 

Teachers  used  to  be  the  only  source  to  knowledge  and  information.  Advanced  technology  allows  learners 
to  discover  and  construct  knowledge  by  themselves.  An  elementary  school  teacher  provided  insight  relative  to  this 
issue:  “I  found  lectures  cannot  solve  my  questions  completely.  That  is  why  I spare  more  room  for  students  to  explore 
and  discover  the  answers  to  their  own  questions.  ” 

To  assist  learners  in  discovering  and  constructing  knowledge.  Dr.  Margaret  Chang  employs  a variety  of 
strategies.  She  believes: 

“I  taught  them  how  to  write  a research  proposal  with  an  abstract.  I ask  them  to  use  the  methods  we  have 
talked  during  the  class.  They  can  pick  whatever  topic  that  interests  them.  They  have  to  discuss  their 
thinking  with  other  senior  students  and  graduate  students  or  their  instructors.  ” 

Measurement  and  evaluation 

Affected  by  the  previous  key  points,  measurement  and  evaluation  strategies  are  being  altered.  Traditional 
measurement  like  paper  and  pen  tests  is  losing  its  dominance.  Alternative  assessment  is  coming  to  the  surface.  Dr. 
Sofen  Chen,  commented  on  this  topic: 

“In  my  class,  I used  a lot  of  group  discussion  and  a lot  of  team  work.  I probably  only  use  part  of  the  class 
time.  For  the  rest  of  the  class  time,  they  discuss  and  present.  I used  syllabi  from  the  States  and  they  helped 
me  a lot.  Those  materials  influenced  my  measurement  and  evaluation.  ” 

Western  influences  (i.e.,  syllabi)  become  salient  in  instructional  pedagogy  of  the  East  in  this  case.  Students 
are  measured  at  both  individual  and  team  levels.  Collaboration  and  coordination  were  foreign  to  the  culture  of  long 
ago,  but  now  the  antecedents  of  that  ancient  culture  have  been  deeply  immersed  in  the  new  strategies.  We  hope  that 
further  research  in  these  areas  will  bring  together  the  values  of  both  cultures,  for  both  must  be  nurtured. 
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Summary  and  Conclusions 

It  is  hypothesized  that  there  will  emerge  a reflection  of  Confucian  pedagogy  across  all  groups  but  that  its 
importance  will  reflect  the  current  environment  wherein  the  subjects  are  located. 

It  is  hypothesized  that  a value  set  will  emerge  across  groups  that  reflect  Confucian  pedagogy  with  a 
consistent  series  of  key  words  that  identify  specific  teaching  and  learning  attributes.  The  words  and  phrases  we  have 
included  in  the  draft  matrices  are  a partial  reflection  of  the  data  now  being  analyzed. 

The  knowledge  gained  from  the  interviews  will  provide  a past,  present  and  most  likely  future  portrait  of  the 
relationship  between  Confucian  pedagogy  and  the  needs  of  our  UCF  Chinese  students. 

The  conclusions  reached  may  have  parallel  meanings  for  other  students  from  Africa,  Europe  and  Latin 
American  wherein  similar  pedagogical  hierarchies  exist. 

The  research  will  then  provide  the  basis  for  proceeding  with  Stage  II  of  the  research,  assisting  teachers  in 
China  and  Taiwan  in  adapting  to  Ministry  of  Education  edicts  that  English  as  a Second  Language  and  computer 
competencies  be  taught  in  the  public  schools.  This  stage  and  the  ramifications  of  development  of  a symbiotic 
pedagogy  matrix  for  Confucian  concepts  married  to  technology  constructs  will  be  discussed  in  subsequent 
presentations  Indeed,  perhaps  Alishan  Mountain  may  just  be  nudged. . .a  little  bit  at  a time. 

We  view  this  research  as  being  in  its  infant  stages,  with  years  of  further  work  ahead  of  us.  We  are  not 
unmindful  of  the  sensitivity  of  our  undertaking.  We  are  especially  concerned  that  our  actions  do  not  “change  these 
Chinese  students  to  the  extent  that,  upon  their  return  to  their  home  environments,  their  effectiveness  has  been 
culturally  emasculated”  (N.  Ying,  personal  communication,  April  4,  2002).  Conversely,  Chin-Ning  Chu,  in  his 
work.  Thick  Face,  Black  Heart  (1992),  commenting  on  how  one  follows  rules  of  what  is  right  or  wrong,  states: 

We  seek  an  understanding  of  ourselves  so  that  we  will  know  what  we  ought  to  do  in  any  given  situation. 

You  will  gradually  replace  the  beliefs  you  were  taught  with  the  truths  you  discover.  It  is  not  whether  you 

turn  the  other  cheek  that  is  important.  Why  you  do  or  do  not  is  most  significant  (p.37). 

These  two  points  might  appear  diametrically  opposed  to  one  another  but  even  here  there  is  symbiosis  to  the 
one  who  looks  more  deeply  into  these  statements.  We  are  beginning  to  understand  far  more  about  what  our  Chinese 
students  are  thinking  and  how  they  process  the  information  we  provide  them.  The  degree  to  which  we  either 
proceed  toward  cultural/intellectual  emasculation  or  redefine  what  is  “right  or  wrong”  remains  to  be  seen;  indeed, 
our  research  must  be  conducted  with  the  utmost  sensitivity  and  tact;  of  this  we  are  very  well  aware.  Our  proposed 
solutions  are  numerous;  some  have  been  tried  and  proved  helpful;  a few  have  failed;  others  are  being  tested  at  this 
moment. 
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Abstract 

The  field  of  Instructional  Design  is  changing  rapidly,  and  it  is  becoming  increasingly  difficult  for  designers 
to  stay  abreast  of  the  research  and  evaluation  studies  describing  those  changes,  not  to  mention  the  more  informal 
developments  that  arise  from  one  designer  sharing  a new  technique  or  idea  with  others.  A design  team  at  Brigham 
Young  University,  in  cooperation  with  AECT,  is  attempting  to  make  the  latest  practical  knowledge  associated  with 
online  learning  easily  available  to  all  designers  through  the  development  of  an  online  community.  This  paper  will 
discuss  the  origin  of  the  site,  its  development,  its  future,  and  the  daunting  challenge  of  designing  for  designers. 


Introduction 

One  of  the  challenging  aspects  of  the  field  of  Instructional  Design  is  that  it  is  constantly  changing.  Many 
of  these  changes  appear  in  research  articles  or  evaluation  studies  while  others  emerge  in  less  formal  ways,  e.g.,  when 
one  designer  shares  new  techniques  or  ideas  with  another  designer.  Either  way,  it  is  often  extremely  difficult  for 
designers  to  wade  through  all  of  the  available  information  to  find  the  practical  knowledge  they  need  to  complete 
their  projects.  The  research  literature  requires  too  much  time  to  comb  through,  the  cross-institutional  collaboration 
too  much  time  to  develop.  To  stay  abreast  of  the  latest  knowledge  associated  with  online  learning,  instructional 
designers  need  a central  point  of  contact,  a place  where  they  can  go  for  practical  solutions  to  the  problems  they  face 
every  day. 

Such  a place  is  currently  not  available.  There  are  numerous  online  journals  and  discussion  boards,  not  to 
mention  countless  online  course  syllabi  and  instructional  design  tutorials.  However,  none  of  these  has  organized  the 
information  into  a complete  and  usable  community  of  information,  nor  do  they  provide  the  personal  connections  that 
are  most  valuable.  Those  who  have  tried  have  diversified  the  available  information  to  the  point  that  is  no  longer 
practical.  Certainly,  none  has  been  fully  embraced  by  the  international  instructional  design  community.  Thus, 
designers  continue  to  reinvent  processes  and  techniques  that  others  have  already  mastered. 

The  Instructional  Psychology  and  Technology  Department  at  Brigham  Young  University,  in  conjunction 
with  AECT,  is  attempting  to  bridge  the  knowledge  gap  by  creating  an  online  community  of  instructional  designers, 
developers,  and  other  educational  technology  professionals.  The  resulting  site  will  allow  designers  to  share  ideas 
and  techniques  with  and  ask  questions  of  other  designers  and  professionals  from  academia,  the  military,  government 
and  the  corporate  world.  The  site  will  be  a community  where  designers,  developers,  academic  faculty,  students,  and 
others  can  build  relationships  with  their  peers  and  share  valuable  knowledge  with  one  another  regarding  the 
practical,  everyday  implementation  of  instructional  design.  We  have  named  the  site  the  Instructional  Design 
Exchange  Area  (IDEA),  emphasizing  the  fact  that  it  is  to  be  a place  where  ideas  and  practices  are  shared  as  we  strive 
to  improve  our  profession  and  our  field. 

Designing  for  Designers 

The  true  and  daunting  challenge  of  building  this  site  is  that  of  designing  for  designers.  What  information 
will  be  most  useful  for  the  practitioner?  What  tools  will  facilitate  the  interaction  and  relationships  that  the  site  hopes 
to  foster?  How  should  the  site  be  structured  in  order  to  encourage  participation  and  assist  the  practitioner  in  finding 
the  most  valuable  information?  What  colors  and  fonts  should  we  use?  The  last  question  may  seem  funny,  but  it  is  a 
frightening  proposition  when  designing  for  professional  designers,  especially  those  with  a particular  interest  in 
typography! 

Cliff  Eigallo,  a former  leader  of  the  WELL  in  San  Erancisco,  stated  that,  “It’s  my  belief  that  a community 
should  be  a practical  and  useful  thing  for  people  to  join’’  (3).  We  have  attempted  to  make  this  site  as  practical  and 
useful  as  possible,  and  initial  reviews  indicate  that  we  are  on  the  right  track.  We  went  through  several  iterations  to 
get  where  we  are  now,  however.  The  original  idea  for  the  site  centered  on  a discussion  board  in  which  designers 
could  ask  and  answer  questions  about  how  to  do  things.  The  project  snowballed,  as  projects  often  do,  until  the  site 
began  to  look  like  ERIC  with  a discussion  board  on  the  side.  The  argument  for  this  structure  was  that  designers 
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need  access  to  research  in  order  to  be  able  to  ground  their  work  in  sound  instructional  theory.  This  was  a good 
argument,  but  the  resources  to  find  that  research  already  existed  while  this  site  was  supposed  to  be  unique  and 
practical.  Etienne  Wenger,  et  al’s  book.  Cultivating  Communities  of  Practice  dramatically  altered  the  course  of  the 
design  work.  They  said,  “Community  leaders  are  sometimes  tempted  to  strengthen  the  community  by  increasing  the 
amount  of  material  on  their  Web  page,  making  it  a full  library.  Although  populating  a database  is  useful,  feeling 
related  and  responsible  to  other  community  members  is  a far  stronger  force  for  increasing  participation  and 
aliveness”  (Wenger,  133).  We  realized  that  we  needed  to  change  the  structure  of  the  site  in  order  to  make  it  useful 
and  to  encourage  the  development  of  a community. 

Another  design  round  resulted  in  a more  practical  site  which  was  geared  mainly  toward  new  designers  and 
students.  It  contained  best  practices,  some  research  articles,  and  guidelines  for  developing  instructional  products  as 
well  as  areas  for  product  reviews  and  discussion.  After  a review  by  Dr.  Allison  Rossett  of  San  Diego  State 
University,  however,  the  site  was  once  again  revised  to  bring  innovation  and  the  more  practical  features  to  the 
forefront.  This  design  and  revise  cycle  has  concluded  in  a site  that  we  feel  is  very  practical  and  very  useful. 

A total  of  27  individuals  have  reviewed  the  prototype  of  the  site  that  is  currently  online.  Their  reviews  and 
comments  are  encouraging  and  we  have  received  some  excellent  suggestions.  When  asked  how  useful  the  site 
would  be  to  them  on  a 4-point  scale  (1  -not  at  all,  4-extremely)  the  average  rating  was  3.3.  Using  the  same  scale,  the 
site  received  an  average  rating  of  3 for  innovation.  While  we  recognize  that  there  are  definite  improvements  to  be 
made,  these  results  encourage  the  team  to  proceed  in  developing  the  site.  The  following  quote  from  one  site 
reviewer  identifies  both  the  enthusiasm  and  the  concern  of  prospective  users:  “If  you  stick  to  your  focus  of 
instructional  design  for  the  web-  emphasizing  problems  and  solutions  - you'll  definitely  be  high  up  on  my  Favorites 
list.” 

Being  that  IDEA  will  be  a community  for  Instructional  Designers,  the  design  team  felt  that  it  was  necessary 
to  follow  an  instructional  systems  design  model.  The  team  developed  a protocol  that  deviates  slightly  from  the 
standard  ADDIE  model,  calling  it  the  ADEPT  model  of  instructional  design.  This  model  includes  analysis,  design, 
evaluation  of  a prototype,  production,  and  testing  and  implementation.  The  team  felt  that  the  inclusion  of  a 
prototype  phase  was  vital  not  only  for  IDEA  but  for  many  instructional  design  projects.  At  the  time  of  writing,  the 
team  has  worked  through  the  analysis  and  design  phases  and  is  completing  the  evaluation  of  the  prototype.  The 
development  of  the  site  will  be  cyclical,  with  further  evaluation  of  a more  advanced  prototype  resulting  from  the 
work  done  thus  far. 

The  team  also  researched  the  development  of  online  communities  to  ensure  that  accepted  methods  and  tools 
were  considered.  The  remainder  of  this  paper  will  follow  the  definition  of  community  given  by  Dr.  Jenny  Preece: 

“An  online  community  consists  of: 

• People,  who  interact  socially  as  they  strive  to  satisfy  their  own  needs  or  perform  special  roles, 
such  as  leading  or  moderating. 

• A shared  purpose,  such  as  an  interest,  need,  information  exchange,  or  service  that  provides  a 
reason  for  the  community. 

• Policies,  in  the  form  of  tacit  assumptions,  rituals,  protocols,  rules,  and  laws  that  guide  people’s 
interactions. 

• Computer  systems,  to  support  and  mediate  social  interaction  and  facilitate  a sense  of  togetherness’’ 

(10). 

Each  of  these  community  components  will  be  discussed  in  detail  below  including  how  they  are  being  cultivated  in 
the  IDEA  site. 

People 

While  academic  faculty,  developers,  administrators  and  others  are  welcome  to  participate  and  will  find  the 
site  useful,  IDEA  is  intended  for  practicing  Instructional  Designers.  This  distinction  was  made  in  order  to  design  the 
site  to  be  as  useful  and  practical  as  possible.  The  idea  for  the  site  came  from  a realization  that  designers  are  having 
to  reinvent  the  wheel  as  they  attempt  to  design  the  best  possible  online  learning.  Lacking  a central  location  to  ask 
questions,  discuss  ideas  and  see  examples  of  others’  work,  designers  work  independently  and,  often,  parallel  to  each 
other  in  solving  instructional  and  technological  issues.  IDEA  will  provide  Instructional  Designers  with  a 
community  where  they  can  communicate  with  each  other  and  assist  one  another  in  solving  problems  and  working 
with  new  technologies. 

The  first  step  after  deciding  whom  the  target  audience  would  be  was  to  determine  what  their  needs  were 
and  whether  or  not  they  were  interested  in  the  site  concept.  As  stated  by  Figallo,  “Knowing  the  needs  of  your  target 
users  is  where  your  design  work  should  begin’’  (148).  A brief  online  survey  was  drafted  to  determine  how 
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professionals  in  the  field  spend  their  time,  whether  or  not  they  were  interested  in  the  site,  and  what  types  of 
information  they  might  like  to  have  available. 

Instructional  design  professionals  in  both  academic  and  corporate  settings  were  invited  to  participate  in  this 
survey.  A total  of  307  invitations  to  participate  were  distributed  via  email  to  one  representative  at  each  of  145 
companies  and  33  higher  education  institutions,  along  with  65  alumni  of  Brigham  Young  University’s  Instructional 
Psychology  and  Technology  Department  and  64  alumni  of  Utah  State  University’s  Instructional  Technology 
Department.  The  invitational  email  message  included  a description  of  the  project  and  an  invitation  to  forward  the 
survey  to  co-workers  and  others  in  the  field.  Hence  an  exact  return  rate  is  difficult  to  determine.  We  estimate  that 
30-40%  of  those  who  received  the  message  actually  responded. 

A total  of  142  individuals  completed  the  survey.  Respondents  represented  the  full  spectrum  of  experience 
in  the  field,  from  0 to  over  20  years  of  association.  The  average  length  of  time  spent  as  an  instructional  designer 
was  7 years,  with  10  respondents  indicating  over  20  years  of  experience  in  the  field. 

Across  all  participants,  the  most  time  is  spent  in  original  design  work  (23%).  The  next  largest  percent  of 
time  (22%)  is  spent  in  project  management  or  administrative  responsibilities,  followed  by  meetings  (14%)  and 
research  (12%).  Additional  time  is  spent  in  product  reviews,  development,  teaching  and  overhead  functions. 
Responsibilities  in  marketing  and  consulting  were  also  mentioned.  A very  small  proportion  of  time  (2%)  was 
attributed  to  “other”  responsibilities  including  analysis,  implementation,  and  evaluation.  Of  course,  the  above 
numbers  are  an  average  of  the  time  spent  by  all  participants.  The  numbers  vary  slightly  when  divided  by  job  title. 
Instructional  Designers  spend  a larger  percentage  of  their  time  doing  original  design  work,  while  Faculty  spend 
more  time  researching  and  Administrators  listed  supervisory  responsibilities  and  other  associated  duties  as  receiving 
the  majority  of  their  attention. 

Next,  participants  were  asked  to  rate  10  types  of  information  based  on  their  usefulness  on  a standard  Likert 
scale  — 1 being  not  helpful,  5 being  very  helpful.  Respondents  indicated  that  best  practices  (mean  rating  4.2), 
research  articles  (4.1),  and  web  links  (4.0)  would  be  the  most  helpful  with  online  discussion  and  an  online  display 
case  also  rating  quite  high  (3.9  and  3.8,  respectively).  Least  helpful  by  far  (2.0)  were  hardware  reviews. 

When  asked  which  format  they  would  prefer  the  information  to  be  in  on  the  same  1-5  scale,  a website  was 
rated  highest  (4.6)  and  a newsletter  lowest  (3.1).  Some  participants  suggested  additional  formats,  including  round 
table  discussions,  case  studies,  and  web  casts.  Most  appear  to  prefer  a website  backed  by  a searchable  database. 

The  most  encouraging  information  gleaned  from  the  survey  was  that  referring  to  interest  in  the  site.  Survey 
participants  were  asked  to  indicate  their  willingness  to  review  and/or  test  the  resulting  product  as  well  as  their 
willingness  to  contribute  to  that  product.  A full  85%  indicated  a willingness  to  assist  in  testing,  while  75%  said  they 
would  contribute  to  the  finished  site.  Considering  the  broad  spectrum  of  participants  and  the  constant  demands  on 
their  time,  these  numbers  are  very  impressive  and  encouraging.  They  seem  to  suggest  that  such  a website  is  needed 
in  the  educational  technology  community. 

Overall,  the  reaction  to  the  site  proposal  has  been  quite  positive.  One  participant  responded  to  the  email  in 
the  following  way:  “This  is  an  excellent  idea.  I have  been  researching  the  topic  of  building  a community  of  practice 
on  the  web,  and  have  been  wrestling  with  the  issue  of  how  to  get  people  to  participate  beyond  the  first  few  days  of 
excitement.  Finding  out  what  is  useful  for  people  with  a survey  is  certainly  one  of  the  first  steps !” 

The  main  problem  with  building  an  online  community  for  instructional  designers  is  that  designers’  time  is 
limited.  A central  component  of  a community  is  participation.  “The  community  element  is  critical  to  an  effective 
knowledge  structure.  A community  of  practice  is  not  just  a Web  site,  a database,  or  a collection  of  best  practices.  It 
is  a group  of  people  who  interact,  learn  together,  build  relationships,  and  in  the  process  develop  a sense  of  belonging 
and  mutual  commitment”  (Wenger,  34).  By  populating  the  site  with  current  and  useful  information,  providing  a 
network  of  able  peers,  and  allowing  practitioners  to  share  their  insights  and  work  with  others,  we  believe  that 
designers  will  embrace  the  site  and  make  the  community  successful. 

Purpose 

The  Instructional  Design  Exchange  Area  is  intended  to  provide  instructional  design  practitioners  with: 

• a centralized  resource  of  timely,  practical,  and  foundational  information  about  their  field, 

• a forum  for  the  ongoing  discussion  of  ideas  and  solutions  to  real  world  situations, 

• an  opportunity  to  share  the  results  of  their  work, 

• and  access  to  ongoing,  online  Instructional  Systems  Design  briefings  and  instruction. 

IDEA  will  offer  a wide  array  of  tools  and  resources  for  designers.  Eor  those  new  to  the  field,  the  site  will  offer 
guidelines  and  best  practices  to  assist  in  orienting  them  to  the  profession.  Current  research  will  also  be  available  so 
that  designers  can  ground  their  practice  in  theory.  The  site  will  not,  however,  become  a library  of  instructional 
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design  research.  Other  sites,  such  as  ERIC,  perform  that  function  very  well.  Instead,  IDEA  will  focus  on  what’s 
current  and  how  it  affects  the  practice  of  instructional  design. 

Most  importantly,  it  will  be  a “place”  where  designers  can  gather  to  discuss  ideas  and  difficulties  and  to  build 
relationships  with  others  in  the  field.  While  information  the  developers  post  on  the  site  will  be  useful,  it  is  not  the 
lifeblood  of  the  community.  “[W]hat  energizes  the  potential  community  is  the  discovery  that  other  people  face 
similar  problems,  share  a passion  for  the  same  topics,  have  data,  tools,  and  approaches  they  can  contribute,  and  have 
valuable  insights  they  can  learn  from  each  other”  (Wenger,  71). 

The  site  focuses  on  three  areas:  Innovations  in  Online  Learning,  Eorums  & Event,  and  Tools.  These  areas  will 
be  the  center  of  activity  for  the  community.  The  first  is  Innovations  in  Online  Learning.  The  purpose  of  this  section 
is  to  provide  a forum  where  site  members  can  display  their  work,  offer  insights  about  new  and  different  ways  to 
perform  instructional  design  tasks,  and  discuss  the  leading  edge  of  the  field.  It  will  include  a section  for 
“Innovators”  where  individuals  in  the  field  who  have  demonstrated  leadership  and  creativity  will  be  profiled. 

Cutting  edge  research  will  also  be  available  in  this  section,  and  designers  will  be  able  to  showcase  their  own 
innovative  projects  in  the  online  display  case.  New  design  and  development  tools  will  also  be  reviewed  and  links  to 
other  resources  will  be  available. 

Eorums  and  Events  will  be  the  community  message  board.  In  the  forums  area,  designers  will  be  able  to  ask 
questions  and  discuss  the  latest  hot  topics  in  the  field  of  Instructional  Technology.  They  will  have  the  opportunity  to 
make  contacts  and  build  relationships  with  peers  all  over  the  world.  The  need  to  reinvent  the  wheel  will  be  reduced 
as  designers  share  their  strategies  and  solutions  with  one  another.  The  events  area  will  provide  a calendar  of 
activities  not  just  for  the  IDEA  community  but  for  the  field  at  large.  Online  chats  and  webcasts  will  be  listed 
alongside  international  conferences,  calls  for  papers  and  seminars. 

The  Tools  section  will  contain  descriptions,  reviews,  and  forums  of  the  latest  software  tools  in  the  industry, 
allowing  designers  to  learn  about  new  software  and  other  technology  tools  that  may  improve  their  projects. 

Designers  will  also  be  able  to  share  ways  in  which  they  have  used  or  manipulated  tools  to  solve  their  design 
problems.  This  area  will  be  a very  valuable  resource  as  designers  consider  purchasing  options,  particularly  as  we 
move  toward  standardization. 

There  are  three  other  sections  of  the  site  that  are  more  focused  on  the  basic  principles  of  instructional  design. 

The  first  is  Instructional  Systems  Design.  This  section  of  the  site  is  intended  to  provide  best  practice  guidelines, 
tools,  examples  and  peer  advice  to  support  the  Instructional  Systems  Design  and  Development  Lifecycle.  It  follows 
the  ADEPT  model  of  instructional  design  mentioned  above,  but  the  information  is  relevant  to  all  design  models. 

This  is  an  excellent  resource  for  new  Instructional  Designers,  particularly  those  without  formal  training  in  the  field. 
The  section  titled  Applied  Theory  and  Research  will  provide  centralized  access  to  research  and  thinking  relevant  to 
the  field  in  order  for  practitioners  to  be  able  to  ground  their  practice  in  foundational  principles.  It  is  also  intended  to 
act  as  a focal  point  for  identifying  applied  examples,  expanding  and  clarifying  current  research,  identifying  needed 
research,  and  forming  research  collaborations.  The  final  section.  Evaluation,  strives  to  encourage  designers  to 
implement  evaluation  practices  into  their  projects  by  providing  guidelines  and  tools. 

Policies 

All  sections  of  the  site  are  intended  to  be  open  to  growth  and  change.  We  have  already  received 
suggestions  for  expansion  and  clarification  and  the  site  will  continue  to  develop  with  the  field.  While  the  site  will 
receive  support  from  Brigham  Young  University  and  the  Association  for  Educational  Communication  and 
Technology,  much  of  the  responsibility  for  contributing  content  and  ideas  for  expansion  will  rest  on  the  users.  This 
decision  was  made  for  two  reasons.  Eirst,  “If  content  is  supplied  solely  by  the  site  provider,  even  if  it  serves  focused 
interests,  it  loses  its  claim  to  community”  (Eigallo,  45).  The  site  was  intended  to  bring  practitioners  together  to 
solve  common  problems  and  share  cutting  edge  information.  If  the  developers  were  to  provide  all  of  the 
information,  designers  would  not  feel  a need  to  share  their  knowledge  and  expertise  and  the  site  would  fail. 

Secondly,  the  practitioners  are  the  ones  who  really  know  the  issues  of  concern  in  the  field.  They 
experience  the  real-world  implications  of  technological  advancements  and  theory  implementation  every  day.  They 
know  where  the  field  is  and  where  it  is  headed.  Who  better,  then,  to  decide  the  structure  and  content  of  a site  that  is 
meant  to  be  useful  to  them?  The  site  will  allow  two  kinds  of  opportunities  for  practitioners  to  contribute  to  the  site. 

The  first  and  most  basic  way  for  practitioners  to  contribute  to  the  site  will  be  to  submit  information.  This 
information  can  range  anywhere  from  comments  on  the  discussion  board  to  suggestions  for  new  content  areas  to 
research  and  project  demonstrations.  All  areas  of  the  site  will  be  open  for  users  to  contribute  relevant  content.  “By 
assigning  responsibility  to  the  practitioners  themselves  to  generate  and  share  the  knowledge  they  need,  communities 
provide  a social  forum  that  supports  the  living  nature  of  knowledge”  (Wenger,  12). 
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For  those  willing  to  become  more  involved  in  the  site,  they  will  have  the  opportunity  to  take  stewardship 
over  certain  areas  of  interest.  These  “content  stewards”  will  be  responsible  for  running  discussion  boards, 
organizing  site  content,  soliciting  informational  contributions,  and  organizing  events.  Wenger,  et  al,  stated  that  “All 
communities  of  practice  depend  on  internal  leadership,  but  healthy  communities  do  not  depend  entirely  on  the 
leadership  of  one  person.  Leadership  is  distributed  and  is  a characteristic  of  the  whole  community”  (36).  By 
allowing  the  practitioners  themselves  to  have  leadership  roles,  we  hope  to  encourage  participation  and  the  constant 
evolution  of  the  site  to  fit  the  needs  of  the  designers. 

By  keeping  the  content  current  and  relevant  to  the  designers,  the  site  will  continue  to  be  useful  and  the 
community  will  grow.  As  stated  by  Figallo,  “Content  that  doesn’t  change  with  the  times  and  the  sophistication  of 
the  audience  will  soon  be  ignored.  But  content  that  is  updated  and  improved  to  meet  the  changing  needs  and 
increasing  knowledge  of  the  audience  will  remain  a compelling  attraction”  (92). 

Systems 

Wenger  said  that,  “What  allows  members  to  share  knowledge  is  not  the  choice  of  a specific  form  of 
communication  (face-to-face  as  opposed  to  Web-based,  for  instance),  but  the  existence  of  a shared  practice — a 
common  set  of  situations,  problems,  and  perspectives”  (25).  While  this  assumption  is  at  the  heart  of  the  site’s 
purpose,  it  is  also  true  that  the  proper  tools  can  facilitate  and  encourage  purposeful  participation.  The  design  team  is 
currently  researching  the  best  tools  to  use  in  developing  the  actual  site.  The  existing  prototype  demonstrates  the 
content  structure  and  possible  functionality  of  the  site  but  is  not  yet  operational  in  most  of  the  core  capabilities  such 
as  discussion,  search  and  data  storage. 

The  choice  of  a discussion  tool  has  been  one  of  the  hardest  to  make  as  there  are  literally  hundreds,  if  not 
thousands,  of  products  available,  all  with  different  feature  sets  and  pricing  scales.  The  team  is  also  considering 
building  a new  tool  a la  Slashdot  that  will  have  all  of  the  required  functionality.  A more  time  - and  cost-effective 
approach,  however,  seems  to  be  to  adapt  an  existing  system  for  use  within  the  Exchange  Area.  Core  requirements  of 
the  forum  tool  (many  of  which  are  reflected  in  the  rest  of  the  site)  include  self-registration,  a content  rating  system, 
distributable  moderator  features,  and  the  ability  to  set  preferences.  The  leading  contender  at  the  moment  is 
DiscusPro,  but  the  team  has  not  yet  had  the  opportunity  to  test  this  tool  and  therefore  has  not  made  a final  decision 
on  its  use. 

One  element  that  the  developers  feel  is  essential  for  the  success  of  the  community  is  an  efficient  search 
mechanism  so  that  practitioners  can  find  the  information  they  need  quickly.  A generous  donation  by  WexTech  has 
provided  the  site  with  AnswerWorks,  a cutting  edge  real-language  search  engine.  All  site  content  will  be  defined 
working  within  the  framework  of  AnswerWorks  technology  in  order  to  maximize  the  tool’s  ability  to  automatically 
generate  content  summaries  and  FAQs.  This  tool  will  make  all  of  the  information  on  the  site  easily  accessible  for 
practitioners. 

Other  system  considerations  include  the  type  of  server  and  database  to  use  and  whether  or  not  tools  for 
synchronous  communication  (chat  and  webcasts)  and  tailored  content  suggestions  based  on  interest  profiles  will  be 
added.  All  of  these  decisions  will  be  made  when  the  development  team  is  complete. 

Conclusion 

Designing  the  IDEA  community  has  been  an  interesting  and  exciting  challenge.  Creating  a website  for  any 
professional  group  is  a difficult  task,  but  designing  for  designers  has  proved  to  be  both  rewarding  and  nerve- 
wracking.  The  IDEA  design  team  has  attempted  to  follow  both  a sound  instructional  design  model  and  an  accepted 
protocol  for  online  community  development.  Issues  such  as  typography,  chunking,  sequencing,  and  motivation  have 
also  been  considered  in  depth.  While  these  issues  may  be  common  in  website  design,  they  assume  greater 
significance  when  the  site  audience  specializes  in  them.  The  team  has  received  both  criticism  and  praise  in  all  of 
these  areas,  and  we  are  encouraged  to  proceed  in  developing  the  site.  Many  of  those  who  have  had  the  opportunity 
to  review  the  site  have  expressed  appreciation  for  inclusion  in  the  review  process  and  we  hope  that  all  practitioners 
will  embrace  the  opportunity  to  make  this  site  useful  and  interesting.  Eurthermore,  while  we  have  received  many 
suggestions  for  improvement,  the  overall  reaction  to  the  community  has  been  positive.  Look  for  a fully  operational 
version  of  the  site  in  late  spring  2003. 
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Constructing  Learners  in  3D:  An  Investigation  of  Design  Affordances  and 
Constraints  of  Active  Worlds  Educational  Universe 
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Introduction 

During  the  past  decade,  a proliferation  of  new  and  emerging  technologies  have  been  designed  and  adapted  as 
environments  for  distance  learning.  Among  the  more  interesting  contenders  of  adapted  technologies  are  three- 
dimensional  (3D)  virtual  worlds.  Three-dimensional  virtual  worlds  can  be  roughly  described  as  networked  desktop 
virtual  reality.  While  there  are  a variety  of  applications,  typically  most  provide  three  important  features:  the  illusion 
of  3D  space,  avatars  that  serve  as  visual  representations  of  users,  and  an  interactive  chat  environment  for  users  to 
communicate  with  one  another.  Although  most  3D  virtual  worlds  have  been  designed  primarily  as  social  and  gaming 
environments,  one  3D  virtual  world  application.  Active  Worlds,  has  delved  into  the  educational  arena  by  creating 
the  3D  virtual  universe.  Active  Worlds  Educational  Universe,  devoted  solely  to  education. 

While  3D  virtual  worlds  are  a relatively  new  technology  only  recently  being  explored  for  use  as  distance 
learning  environments,  studies  of  text  -based  or  chat  virtual  communities  have  provided  compelling  views  of  how 
technology  influences  the  representation  and  social  construction  of  users  in  a computer-mediated  environment 
(Bmckman,  1997;  Riner  1996;  Turkle,  1995).  Like  text -based  chat  communities,  3D  virtual  worlds  provide  users 
with  a sense  of  presence.  Participants  can  engage  in  conversation,  build,  and  interact  within  an  environment.  Unlike 
text -based  chat  environments,  virtual  worlds  afford  users  a visual  representation  of  place,  space,  and  self  that 
provides  users  with  a sense  of  immersion  and  embodiment  in  the  environment.  While  virtual  worlds  do  provide  a 
sense  of  embodiment,  it  is  important  to  note  that  this  embodiment  may  be  encoded  with  values  and  beliefs  not 
readily  apparent  to  users . Research  into  race,  ethnicity,  culture,  and  gender  in  virtual  communities  argues  that 
virtual  communities  are  embedded  with  values  and  hidden  assumptions  that  promote  the  interests  of  some  users, 
while  marginalizing  other  users  (Bailey,  1996;  Balsamo,  1994;  Branwyn,  1994;  Cherny,  1995;  Dibbell,  1994; 
Milthorp,  1996;  Morningstar  and  Farmer,  1994;  Stone,  1995;  Todd  1996;  Turkle  1995).  While  new  technologies 
offer  more  options  for  distance  learning,  it  is  important  that  researchers,  educators,  and  practitioners  be  aware  of 
how  an  application  may  construct  a user/learner  through  the  values  and  hidden  assumptions  encoded  within  an 
application  (DeVaney,  1993). 


Purpose  of  Study 

The  purpose  of  this  study  is  to  investigate  how  the  design  affordances  and  constraints  of  Active  Worlds 
Educational  Universe  impact  the  construction  of  learners  and  the  culture  of  an  educational  three-dimensional  virtual 
world  setting.  Specifically  this  investigation  will  examine  affordances  and  constraints  that  impact:  (a)  presence;  (b) 
individual  and  collective  representation;  and  (c)  embodiment.  The  context  of  this  study  is  Active  Worlds 
Educational  Universe,  a three-dimensional  virtual  world  universe  created  and  supported  by  the  developers  of  Active 
Worlds.  The  goal  of  this  study  is  to  reveal  how  values  and  assumptions  embedded  in  the  design  of  an  educational 
application  impact  the  presence,  representation,  and  embodiment  of  learners. 

Theoretical  Framework 

The  underlying  question  that  guides  this  investigation  is  how  do  educational  computer  programs  construct  their 
subjects?  Ann  DeVaney  (1993)  first  posed  this  question  in  an  article  entitled,  Reading  Educational  Computer 
Programs.  According  to  DeVaney,  questions  such  as  this  are  important  for  addressing  often  neglected  culture -bound 
issues  of  educational  technology.  In  an  era  of  diversity  and  inclusion,  it  is  important  the  educators  be  vigilant  in 
addressing  ways  in  which  educational  technology  predispose  learners  to  certain  actions  which  may  valorize  some 
learners  and  depreciate  others. 
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Methodology 

The  methodological  framework  employed  in  this  investigation  is  a liberal  adaptation  of  Affordance  Theory. 
Affordance  Theory,  as  described  by  ecological  psychologist,  James  Gibson  (1977),  expresses  the  relationship  that 
exists  between  an  animal  (perceiver)  and  the  environment  (perceived).  The  relevance  of  liberally  adapting 
Affordance  Theory  for  an  analysis  of  a 3D  virtual  worlds  setting  is  that  3D  virtual  worlds  are  simulated 
environments.  The  affordances  and  constraints  of  the  application  impact  opportunities  for  how  an  environment  may 
be  perceived. 

Methods  employed  in  this  qualitative  investigation  include  participatory  observations  (Adler  and  Adler,  1994), 
informal  interviews  (Fontana  and  Frey,  1994)  with  learners,  developers,  and  educators  of  Active  Worlds  Educational 
Universe,  and  interactions  and  observations  with  my  students  using  Active  Worlds  Educational  Universe. 

Setting 

The  setting  for  this  investigation  is  Active  Worlds  Educational  Universe.  Active  Worlds  first  premiered  in 
1995.  The  client-server  application  consists  of  the  Active  Worlds  Universe  with  over  1000  individual  worlds  for 
users  to  interact  with  other  users  worldwide.  In  1999,  the  owners  of  Active  Worlds  created  the  Active  Worlds 
Educational  Universe,  a universe  devoted  solely  to  education  initiatives.  The  Active  Worlds  Educational  Universe 
(AWEDU)  affords  user-extensible  provisions  and  support  for  building  new  worlds  and  adding  to  existing  worlds. 
World  owners  are  free  to  define  and  customize  their  world  in  whichever  way  they  choose  by  selecting  objects  from 
the  AWEDU  object  library  or  by  adding  custom  built  objects. 

The  AWEDU  browser  interface  is  comprised  of  four  main  scalable  windows  which  include  a 3D 
environment;  a chat  dialogue  window;  an  integrated  web  browser;  and  a window  for  added  navigational  and 
communicational  functions  (see  Eigure  1).  The  3D  environment  is  the  primary  setting  for  interaction.  Learners 
represented  as  avatars,  move  and  interact  with  each  other  and  the  environment  by  either  moving  their  mouse  or  by 
using  arrow  keys  on  their  keyboard.  Directly  beneath  the  3D  environment  is  the  chat  window.  Communication  is 
limited  to  text  messages  which  are  display  both  above  the  speaker’s  avatar  in  the  3D  environment  and  in  the  chat 
window  below. 

Eigure  1.  The  Active  Worlds  Educational  Universe  browser. 


Findings  and  Analysis:  Presence/Attendance 

Initially  this  investigation  examined  how  the  learner  was  constructed  by  focusing  on  three  categories: 
presence,  representation,  and  embodiment.  Initially  presence  was  determined  by  looking  at  the  affordances  and 
constraints  of  hardware  and  software  requirements,  skills  necessary  to  participate,  and  language  accessibility. 
Flowever,  as  this  study  progressed,  the  categories  that  are  emerging  are  not  as  first  believed,  but  rather 
representation  and  embodiment  are  all  elements  that  construct  the  learners  Presence.  The  category  initially  labeled 
as  presence  now  seems  more  aptly  described  as  attendance. 
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Findings 

There  are  various  affordances  and  constraints  that  impact  a learner’s  attendance  in  the  world.  Hardware 
requirements  include:  Pentium  200mhz  or  equivalent,  64MB  RAM,  Microsoft  Windows  (95-  XP),  and  DirectX  3.0. 
Currently  there  is  no  MAC  version;  however.  Active  Worlds  Educational  Universe  is  accessible  on  a Mac  running 
Virtual  PC.  While  there  are  few  skills  required  to  participate  within  the  Active  Worlds  Educational  Universe,  basic 
skills  required  would  include  the  ability  or  access  to  someone  who  can  download  the  application  from  a website  and 
install  it. 

Learners  are  constructed  by  language  both  in  AWEDU  and  in  the  documentation.  AWEDU  is  one  of  the 
few  3D  virtual  worlds  to  support  many  languages.  While  English  is  most  often  observed  in  some  of  the  more 
commonly  visited  worlds,  it  is  by  no  means  the  only  language  used.  Depending  upon  the  time  of  day  and  location, 
English  may  not  be  observed  being  used  at  all.  The  AWEDU  browser  supports  the  following  languages:  Spanish, 
Danish,  English,  Dutch,  Erench,  German,  Einnish,  Italian,  Hungarian,  Norwegian,  Portuguese,  and  Swedish.  Despite 
language  support,  some  of  the  browser  features  remain  in  English  (Figure  2.).  English  is  also  the  primary  language 
used  for  documentation.  While  there  are  provisions  for  multi-language  support,  clearly  some  knowledge  of  English 
would  be  an  advantage  for  learners  in  this  environment. 

While  the  AWEDU  browser  officially  only  supports  a select  list  of  languages,  it  is  not  uncommon  to  find 
many  other  languages  being  used  in  various  worlds  including  both  Japanese  and  Russian.  Additionally,  since  many 
languages  use  the  same  alphanumeric  system  as  is  used  in  English,  many  more  languages  are  used  than  are 
represented  by  the  browser. 

Figure  2. The  Italian  language  version  of  the  AWEDU  browser. 


Analysis 

Learner  attendance  is  constructed  by  way  of  the  hardware,  software,  downloading  skills,  and  language. 
Learners  must  have  access  to  a Pentium  computer  and  have  Internet  access.  In  addition  learners  must  possess  the 
skills  to  be  able  to  download  an  application  and  load  it  on  a machine.  This  means  the  learner  must  also  have  the 
language  skills  to  read  the  instructions  or  have  access  to  a translation. 

Learners’  construction  is  impacted  by  language  accessibility  both  in  AWEDU  and  by  access  to 
documentation.  Although  the  AWEDU  browser  supports  multiple  languages,  the  languages  supported  reflect 
primarily  Western,  European  languages.  Equally  important  to  note  is  that  not  all  functions  are  translated.  Learners 
still  to  some  extent  must  have  English  language  skills  or  have  access  to  a translation. 


Findings  and  Analysis:  Individual  and  Collective  Representation 

In  AWEDU  learner  representation  is  constructed  in  various  ways.  The  identity  of  the  learner  is  essential 
forrecognition,  while  avatars  serve  as  the  visual  representation  of  learners  in  the  3D  environment.  While  these 
twoelements  comprise  and  impact  the  direct  construction  of  the  learner,  the  environment  also  plays  a strong  role  by 
illustrating  world  views  displayed  in  the  setting. 
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Findings:  Identity 

In  real  life,  identity  is  often  tied  to  some  physical  aspect  of  an  individual.  Appearance,  facial  features  and  voice 
are  among  the  means  by  which  we  distinguish  one  another.  In  the  AWEDU  environment,  these  physical  cues  are  not 
available.  Learners  are  not  recognized  by  voice,  or  by  their  appearance,  but  instead  must  rely  on  unique  identities  to 
identify  one  another.  Within  the  Active  Worlds  Educational  Universe  registered  users  may  select  a unique  identity. 
This  identity/name  may  not  be  used  by  any  other  leaner.  This  provision  allows  learners  to  recognize  and  contact 
each  other.  A learner’s  unique  name  appears  above  his/her  avatar  the  first  time  he/she  uses  the  chat  tool  during  a 
session.  If  the  learner  wishes  to  remain  anonymous  or  lurk,  not  “speaking”  will  insure  anonymity. 

With  a unique  identity,  learners  may  have  extended  communication  options  such  maintaining  a contact  list 
which  allows  them  to  identify  when  another  learner  on  their  list  is  currently  active.  The  may  also  send  and  receive 
telegrams  from  learners  currently  inhabiting  other  worlds.  Although  learners  have  the  option  of  changing  their 
unique  name,  any  name  change  will  be  reflected  in  other  learner’s  contact  lists. 

Analysis:  Identity 

A unique  identity  brings  both  privileges  and  accountability.  It  allows  learners  to  establish  both  reliability  and 
consistency  in  both  the  personal  and  social  arena.  Learners  can  build,  develop  contact  lists,  send  and  receive 
telegrams.  This  is  important  for  establishing  a degree  of  trust  among  learners.  Along  with  the  privileges  of  a unique 
identity  comes  a degree  of  accountability.  Within  a system  where  learners  adopt  alias  identities  a unique  identity 
prevents  learners  from  impersonating  one  another.  In  turn,  this  affords  learners  with  some  degree  of  self-governance 
among  learners.  The  accountability  established  by  a unique  identity  can  also  impose  inverse  limits  on  well  meaning 
learners.  Once  a learner  establishes  a unique  identity  and  other  learners  may  add  him/her  to  their  contact  list,  any 
subsequent  changes  in  his/her  adopted  identity  will  be  reflected  on  the  other  learners’  contact  list.  Initially  this  may 
not  seem  to  impose  limitations  on  how  the  learners  represents  themselves;  however,  in  an  environment  where 
learners  are  free  to  re  -construct  themselves  or  develop  alternate  personae  this  can  prove  constrictive. 

Findings:  Avatars 

The  creators  of  AWEDU  define  ''avatar''  as  "the  visual  representation  of  people  who  currently  inhabit  the 
virtual  environment.  In  an  AWEDU  world,  avatars  serve  not  only  as  the  visual  representation  of  a learner,  but  also 
as  the  camera  into  a 3D  environment. 

AWEDU  provides  a stable  of  avatars  that  world  owners  may  select  and  make  available  to  learners  visiting  a 
world  setting.  Learners  are  limited  to  selecting  from  avatars  provided  by  world  owners.  Because  of  the  limited 
range  of  avatars,  many  learners  may  use  the  same  avatar  within  an  environment.  Consequently,  learners  must  rely 
upon  the  unique  identify  to  identify  other  learners. 

With  few  exceptions  the  AWEDU  avatars  represent  young,  fit.  Western,  Caucasians.  There  are  more  male 
avatars  than  female,  and  it  is  interesting  to  note  that  there  is  a greater  range  of  body  types,  sizes,  ages,  and  styles  of 
male  avatars  than  female.  Within  the  selection  of  female  avatars,  there  is  little  variety  in  size,  weight,  age,  ethnicity, 
and  race.  Most  of  the  female  avatars  represent  young  shapely  women  with  either  snugly  fitting  clothing  or  short 
dresses  and  high  heels.  There  are  also  no  avatars  representing  differently -abled  persons.  There  are,  however,  a few 
alternatives  for  learners  not  wanting  to  be  represented  as  humans  or  as  a gender  typically  in  the  form  of  a bird  or 
alien-type  caricature. 

AWEDU  allows  world  owners  to  create  and  import  custom-made  avatars;  however,  creating  an  AWEDU  avatar 
is  no  small  feat,  nor  one  easily  accessible  to  many  learners.  Creating  an  AWEDU  avatar  requires  a basic  knowledge 
of  3D  concepts.  It  also  requires  that  a learner  be  fairly  proficient  with  creating  Renderware  (RWX)  objects  or  in 
using  3D  modeling  software  such  as  Caligari’s  Truespace. 

Analysis:  Avatars 

Eor  most  learners  there  is  little  opportunity  for  creating  custom  avatars.  With  the  exception  of  world  owners, 
learners  are  confined  to  using  one  of  the  prefabricated  avatars  provided  by  an  individual  world.  Eor  visual 
representation,  learners  must  rely  on  avatars  that  may  not  reflect  their  personal  values,  culture,  ethnicity,  or  physical 
bodies.  The  avatars  are  primarily  idealized  representations  of  young  adults  that  clearly  reflect  Western  body -image 
values. 

Findings:  Environment 

Initially  it  may  seem  incongruent  to  include  environment  as  a component  in  learner  representation. 
However,  in  AWEDU  while  various  factors  may  limit  or  prevent  learners  from  creating  and  using  custom-made 
avatars,  many  worlds  provide  opportunities  for  building  within  worlds.  AWEDU  is  a user-extensible  system  which 
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allows  learners  to  add  to  and  build  within  a world  setting.  Depending  upon  provisions  determined  by  the  owner  of  a 
world,  learners  can  claim  property  by  placing  ground  covering  on  a section  and  build.  Once  a piece  of  land  is  staked 
or  covered  by  a learner,  no  other  user  may  build  on  that  land,  or  above  and  below  it.  To  build  or  add  to  property 
claim,  a learner  merely  chooses  from  a selection  of  pre -fabricated  objects  that  exists  in  the  object  library  and  places 
the  object  in  the  desired  location.  The  objects  in  the  library  range  from  a variety  of  building  materials  such  as  walls, 
doors,  and  windows  to  specific  items  such  as  televisions,  glasses,  and  cars.  There  are  also  a full  range  of  objects 
such  as  flowers,  trees,  grass,  and  bushes  that  allow  users  to  create  an  outdoor  setting  as  well. 

Analysis:  Environment 

Throughout  most  of  the  worlds  in  the  AWEDU,  are  elaborate  buildings  and  roadways  constructed  by 
learners  and  developers.  These  environments  sometimes  reflect  places  that  exist  in  the  physical  world,  and 
sometimes  border  on  fantastic.  One  question  worth  asking  is  why  is  there  a need  for  buildings  and  roads?  In  life, 
buildings  serve  many  purposes,  but  primarily  they  offer  us  shelter  from  the  environment.  In  a virtual  world  setting, 
there  is  no  impact  from  the  environment.  It  doesn’t  rain,  snow,  and  the  temperature  has  no  impact  upon  our  avatars. 
There  is  no  real  sun  and  the  ambient  lighting  never  changes.  Why  have  buildings?  Why  have  ground  or  gravity? 

The  stock  objects  reflect  and  anticipate  how  developers  perceive  AWEDU  will  be  used.  While  the  provision  of  stock 
objects  affords  an  assessable  means  of  building  in  a world,  it  also  constrains  how  one  might  envision  a world  (Heim, 
1998). 

The  objects  provided  by  AWEDU  also  reflect  values  of  how  nature  is  to  be  regarded.  While  there  are  a plethora 
of  tress,  flowers,  bushes  and  grass  to  select,  it  is  important  to  note  that  there  are  no  weeds,  the  grass  is  well  groomed, 
and  the  trees  never  shed  leaves  or  require  attendance.  Nature  is  contained  and  controlled.  Nature  is  modeled  on  a 
culture  in  which  is  mechanized  and  submissive  (Merchant,  1980).  The  values  perpetuated  clearly  reflect  Western 
values  of  nature  and  the  environment  with  no  mutual  impact  on  either  the  environment  from  the  avatars  or  upon  the 
avatars  from  the  environment.  In  AWEDU,  nature  is  contained  and  subservient,  and  learners  are  both  in  control  and 
impervious  from  impact.  Both  the  types  of  objects  for  constructing  structures  and  the  objects  available  for 
constructing  nature  reflect  also  reflect  Western  values. 


Findings  and  Analysis:  Embodiment 

It  may  seem  somewhat  incongruent  to  speak  of  embodiment  in  a computer- mediated  environment  such  as 
AWEDU.  Although  there  are  avatars  serving  as  representatives  of  learners,  nothing  really  impacts  the  avatar.  It 
doesn’t  get  hurt,  feel  hungry,  or  become  tired.  Avatars  do  not  feel  gravity  or  in  anyway  impact  the  environment. 
Despite  that  lack  of  kinesthetic  cues,  surprisingly  research  reveals  that  users  in  3D  virtual  world  settings  often  adopt 
and  maintain  proximity  customs  from  in  real-world  interactions  (Jeffery  & Mark,  1998).  Embodiment  in  AWEDU  is 
constructed  by  way  of  animated  avatar  gestures,  actions,  and  emotions;  learner  point-of-view  and  navigation;  and 
kinesthetics  and  the  environment. 

Findings:  Gestures/Emotions/Actions 

In  an  AWEDU  world,  avatars  are  rarely  idle.  Unless  the  learner  specifically  alters  the  performance  settings, 
all  avatars  inhabiting  a world  will  cycle  through  a series  of  gestures.  The  gestures  are  not  controlled  by  learners,  but 
instead  are  pre-programmed  animations  which  consist  of  such  movements  as  the  folding  and  unfolding  of  arms, 
looking  from  side  to  side,  looking  at  a wristwatch  (wh  ich  may  not  be  present),  shifting  weight,  tapping  a foot,  and 
patting  hair.  Some  facial  gestures  include  blinking  and  winking.  Learner  have  no  direct  control  over  these  gestures, 
but  are  limited  in  whether  or  not  to  display  them  on  their  own  systems. 

Learners  are  only  able  to  command  a few  emotions  and  actions  of  their  avatar.  The  emotion  commands 
might  include  such  emotions  as  happy  (the  users  avatar  jumps  up  and  down),  and  angry  (pointing  and  shaking  hand). 
Actions  that  can  be  controlled  by  the  user  include:  wave,  jump,  and  fight.  These  actions  are  not  reflected  from  the 
learner’s  perspective. 

Findings:  Avatar  Point -of-View,  Navigation,  and  Kinesthetics 

In  AWEDU,  the  avatar  serves  two  distinct  purposes:  it  serves  as  a visual  representation  of  the  learner  and  it 
serves  as  the  camera  by  which  the  learner  views  the  3D  environment  or  scene.  Learners  have  the  option  of  selecting 
between  first  person  perspective  or  an  orthographic  or  third  person  perspective.  The  orthographic  view  allows  users 
to  see  themselves  (or  more  correctly,  their  visual  representations)  in  3D.  This  feature  is  helpful  for  choosing  an 
avatar  and  building  within  the  world.  In  order  for  the  learners  to  move  through  simulated  3D  space;  it  is  necessary 
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for  the  avatar  or  camera  to  move.  Avatar  navigation  is  limited  to  moving  forward  and  backward  and  side  to  side. 
Learners  may  additionally  look  up  and  down  as  well  as  fly  and  descend. 

There  are  few  affordances  that  provide  for  much  kinesthetic  cues  and  experiences.  Learners’  cannot  control 
avatar  limbs,  and  other  than  navigational  movement,  very  little  is  reflected  from  either  first-person  or  third -person 
perspective.  While  avatars  do  not  register  impact  with  each  other,  depending  upon  specifications  designated  by  a 
world  owner,  avatars  may  register  impact  upon  encountering  an  object. 

Analysis:  Embodiment 

Learner  embodiment  is  constructed  through  gestures,  emotions  and  actions,  navigation  and  viewing 
perspective.  From  the  perspective  of  cultural  kinesthetic  or  non-verbal  behavior  it  is  interesting  to  examine  the 
implication  of  some  of  the  gestures  that  animate  learner  avatars.  In  most  North  American  cultures  when  engaging  in 
a conversation,  looking  at  one’s  watch,  or  looking  from  side  to  side  might  indicate  boredom  or  impatience. 
Additionally,  to  wink  at  someone  may  imply  a range  of  different  meanings  within  the  context  of  a conversation. 
Although  learners  have  no  direct  control  over  gestures,  the  non-verbal  behavior  conveyed  through  gestures  may  be 
in  direct  conflict  to  what  is  being  communicated  through  the  textbox. 

Learners  have  limited  control  over  actions  and  emotions.  They  have  the  option  of  displaying  an  action  or 
emotion;  however  they  have  no  control  over  how  the  action  or  emotion  is  displayed.  None  of  the  animations  are 
reflected  fro  m a first-person  perspective  but  can  only  be  viewed  from  third -person  perspective. 

There  are  few  affordances  for  embodiment  in  the  AWEDU  environment.  For  the  most  part,  learners  are 
prevented  from  displaying  much  non-verbal  communication  but  must  rely  on  text  to  convey  emotions.  This  reliance 
on  text  might  lead  one  to  assume  that  the  user  is  limited  to  text  in  constructing  a sense  of  embodiment.  Embodiment 
to  some  degree  in  AWEDU  is  relegated  to  the  realm  of  navigation  and  proximity.  The  ability  to  see  other  learners 
and  to  move  around  may  be  sufficient  to  construct  at  least  a limited  sense  of  embodiment  for  learners.  However, 
depending  upon  the  purpose  for  which  AWEDU  is  being  used,  it  may  be  important  to  question  how  the  limits  of 
embodiment  imposed  on  the  user  might  limit  the  types  of  experiential  knowledge  that  may  be  gleaned  from  these 
environments. 


Discussion  and  Conclusion 

One  of  the  considerations  that  should  be  addressed  by  using  AWEDU  as  a learning  environment  is  whose 
world  view  is  being  promoted  in  this  environment?  AWEDU  is  embedded  with  cultural  information  and  values. 
Choices  made  by  the  developers  reflect  and  perpetuate  ontological  and  epistemological  views  of  the  developers  and 
in  turn,  learners  in  this  environment  are  constructed  by  cultural  values  that  may  not  reflect  their  own  world  views. 
Research  into  race,  ethnicity,  culture,  and  gender  in  virtual  communities  argues  that  virtual  communities  are 
embedded  with  values  and  hidden  assumptions  that  promote  the  interests  of  some  users,  while  marginalizing  other 
users  (Bailey,  1996;  Balsamo,  1994;  Branwyn,  1994;  Cherny,  1995;  Milthorp,  1996;  Stone,  1995;  Todd  1996; 
Turkle  1995)  and  AWEDU  is  no  exception.  To  enter  this  virtual  environment  requires  one  to  move  into  Cartesian 
space;  to  temporarily  accept  the  notion  of  a mind/body  split  (Pryor  and  Scott,  1993).  This  temporary  acquiescence 
is  a product  of  Western  cultural  values  and  is  not  shared  by  all  cultures  (Todd,  1996).  The  paradox  of  the  move  into 
Cartesian  space  is  that  body  hidden  or  obscured  is  both  liberated  and  repressed  (Stone,  1995).  Learners  are  both 
emancipated  from  physical-bound  representation,  yet  a new  construction  of  self  is  filtered  through  technology. 

AWEDU  is  a product  of  a Western  ontology  which  is  reflected  in  both  how  the  learner  and  the  environment  are 
constructed.  The  affordances  and  constraints  of  the  application  reveal  Western  values  both  in  how  the  learner  and 
nature/environment  are  represented  and  by  the  lack  of  impact  either  have  upon  the  other.  The  purpose  of  this 
investigation  is  not  to  discount  the  potential  AWEDU  might  offer  as  a learning  environment,  but  rather  to  apply  a 
critical-cultural  lens  in  which  to  view  how  this  application  might  impact  potential  learners.  In  an  era  in  which 
inclusion  and  diversity  imbricate  education,  it  is  important  we  challenge  educational  developers  to  rethink  the 
impact  their  designs  have  upon  the  relationship  between  learners  and  technology.  Despite  the  constraints  AWEDU 
imposes  on  learners  one  way  in  way  in  which  this  technology  may  prove  most  educational  is  that  it  a revealing 
mirror  that  reflects  both  values  and  assumptions  about  our  culture  and  all  it  entails. 
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Abstract 

We  have  been  successful  in  prescribing  effective  instructional  strategies  for  learning  in  well-defined 
domains.  Yet,  we  do  not  know  how  to  design  effective  instruction  for  complex  domains.  Scientific  attitude  with 
regard  to  the  design  of  environments  to  support  learning  in  complex  domains  requires  measures  of  learning 
outcomes  in  a variety  of  settings  in  order  to  determine  what  type  of  instruction  work  best,  when,  and  why.  This 
paper  presents  the  results  of  a preliminary  study  about  assessing  progress  of  learning  in  complex  domains.  The 
assessment  methodology  is  based  on  the  use  of  causal  influence  diagrams.  The  domain  of  application  of  the 
methodology  is  instructional  design.  The  paper  concludes  with  a discussion  of  implications  for  instructional  design 
research. 


Introduction 

An  important,  yet  little  understood  question  remains  as  how  to  facilitate  learning  and  promote  expertise  in 
complex  domains.  Our  knowledge  about  the  facilitation  of  learning  has  significantly  increased  in  the  past  two 
decades.  Now,  we  know  quite  well  how  to  facilitate  learning  in  well-defined  domains.  However,  as  the  learning 
goals  and  situations  increase  in  complexity,  our  existing  knowledge  about  the  facilitation  of  learning  becomes 
insufficient.  Part  of  the  problem  is  that  we  do  not  yet  understand  how  people  develop  expertise  in  such  domains 
(Dreyfus  & Dreyfus,  1986;  Ericson  & Smith,  1991). 

A number  of  researchers  increasingly  argue  for  the  need  to  address  issues  pertaining  to  learning  in  and 
about  complex  domains  (Spector  & Anderson,  2000;  Sterman,  1994).  This  is  an  important  issue  because  complex 
systems  exist  in  abundance  in  our  daily  lives.  Economy,  environmental  problems,  the  spread  of  epidemics,  resource 
allocation  in  large  project  environments,  instructional  design,  and  training  are  all  examples  of  complex  domains. 
Eurthermore,  the  tasks  humans  face  in  their  daily  lives  are  becoming  more  and  more  complex  due  to  the 
advancement  of  the  technologies  involved.  Machines  are  already  carrying  the  routine,  procedural  tasks,  which  can 
be  automated.  Therefore,  with  the  advancement  of  technology,  humans  are  now  left  with  the  more  complex  tasks. 

Spector  et  al.  (2001)  argue  that  complex  domains  present  the  most  significant  challenges  for  the  future 
well-being  of  our  species  on  this  planet.  This  is  well  supported  by  Dorner  (1996)  who  states  the  learning  problem 
most  clearly  in  “The  Logic  of  Eailure’’.  He  analyzes  several  disasters  that  involved  poor  human  decision-making  in 
complex  domains,  including  the  Chernobyl  nuclear  reactor  incident  in  Russia.  His  conclusion  of  that  analysis  is  that 
those  involved  were  highly  educated,  well  trained,  and  strongly  motivated  to  do  the  right  thing,  yet,  they  failed.  This 
and  other  examples  in  the  book  clearly  demonstrate  the  need  to  develop  effective  strategies  to  facilitate  learning  and 
promoting  expertise  in  complex  domains. 

Why  is  it  difficult  to  make  good  decisions  in  and  about  complex  domains?  Complex  domains  can  be 
depicted  as  a collection  of  many  interrelated  components  or  variables.  The  relationships  between  these  interrelated 
components  are  mostly  non-linear  and  fuzzy.  Eurthermore,  these  relationships  may  change  over  time  and  may  cause 
delayed  effects  and  uncertainties  due  to  dependency  on  human  perceptions  about  some  aspects  of  the  system 
(Christensen,  Spector,  Sioutine,  & McCormack,  2000;  Sterman,  1988,  1994)  . Therefore,  it  is  difficult  to  understand 
and  deal  with  complex  systems  (Dorner,  1996;  Seel,  Al-Diban,  & Blumschein,  2000).  On  the  other  hand,  simple 
domains  do  not  possess  that  kind  of  dynamic  relationships.  Rather,  a simple  domain  can  be  characterized  by  a 
predominantly  linear  relationship  among  its  components  or  variables.  Therefore,  it  is  much  easier  to  comprehend 
how  a simple  system  works. 

Advances  in  educational  technology  have  led  to  interest  in  providing  meaningful  support  for  learning  about 
complex  domains  (Alessi,  2000;  Spector  & Davidsen,  1997;  Sterman,  1994).  Erom  a research  perspective,  these 
domains  present  the  most  significant  challenges,  both  for  determining  factors,  which  contribute  to  learning  and  for 
designing  effective  learning  environments.  There  are  a number  of  researchers  who  believe  that  we  can  do  better  by 
building  on  graduated  complexity  (see  Spector  & Anderson,  2000).  However,  there  are  no  well-established  methods 
to  assess  learning  outcomes  in  complex  domains.  As  a consequence,  it  is  difficult  to  argue  that  one  approach  is  more 
effective  than  another  in  supporting  learning  in  and  about  complex  learning.  Scientific  attitude  with  regard  to  the 
design  of  environments  to  support  learning  in  complex  domains  requires  measures  of  learning  outcomes  in  a variety 
of  settings  in  order  to  determine  what  type  of  instruction  work  best,  when,  and  why. 
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The  purpose  of  this  paper  is  to  discuss  the  results  of  a preliminary  study  about  assessment  of  learning  in 
complex  domains.  The  goal  of  this  research  effort  was  to  investigate  the  potential  of  causal  influence  diagrams  for 
assessing  progress  of  learning  in  complex  domains.  The  domain  of  investigation  was  instructional  design.  This  study 
was  based  on  four  main  assumptions:  (1)  learning  can  be  characterized  in  part  as  the  acquisition  of  expertise;  (2) 
those  considered  expert  in  a complex  domain  tend  to  produce  similar  causal  influence  diagrams  for  problem 
situations  in  that  domain;  (3)  those  new  to  a complex  domain  tend  to  produce  varied  causal  influence  diagrams  for 
such  problems  that  are  noticeably  different  from  those  of  experts;  and  (4)  different  pedagogical  methods  will  be 
more  or  less  effective  in  producing  desired  learning  outcomes.  The  value  of  this  study  lies  in  its  potential  to 
demonstrate  whether  expert  instructional  designers  create  recognizably  similar  causal  representations  of  common 
problem  scenarios.  If  this  can  be  established  then  these  representations  can  provide  a benchmark  for  learner  progress 
by  comparing  how  learner  representations  change  through  instruction  to  resemble  or  fail  to  resemble  expert  patterns. 
As  a result  we  can  test  the  effectiveness  of  different  instructional  approaches  in  facilitating  learning,  and  thus,  we 
can  contribute  to  understanding  the  development  and  promotion  of  expertise  in  complex  domains. 


Method 

Participants 

Six  expert  instructional  designers  participated  in  this  study.  All  of  them  held  graduate  degrees  (5  with  Ph.D. 
and  1 with  master’s)  in  instructional  design  or  a related  field.  They  were  employed  in  academia  or  in  business.  Mean 
age  of  the  participants  was  55.5  years.  All  of  the  participants  had  experiences  in  practicing  instructional  design  and 
development;  teaching  ID  and  related  courses;  and  conducting  research  in  the  field  of  instructional  design  and 
technology.  The  average  for  full-time  experience  in  practicing  instructional  design  and  development  was  19.83  years 
(min.  10  yrs;  max.  34  yrs.);  for  teaching  ID  related  courses,  it  was  5.33  years  (min.  0 yr;  max.  15  yrs);  and  for  field 
research,  it  was  5.75  years  (min.  1 yr;  max.  18  yrs).  Table  1 presents  frequency  counts  of  instructional  design 
activities  participants  regularly  performed  in  their  current  job. 


Table  1.  Frequency  distribution  of  instructional  design  activities  participants  regularly  performed 


ID  Activity 

Frequency  (N=6) 

Percent 

Conduct  needs  assessment 

3 

0.50 

Determine  solution  alternatives  and 
approaches 

6 

1.00 

Propose  solutions 

4 

0.67 

Write  learning  objectives 

4 

0.67 

Conduct  task  analysis 

2 

0.33 

Identify  types  of  learning  outcomes 

3 

0.50 

Assess  learner’s  entry-level  skills 

3 

0.50 

Assess  learner  characteristics 

5 

0.83 

Develop  test  items 

3 

0.50 

Select  instructional  strategies 

6 

1.00 

Select  media  formats 

5 

0.83 

Conduct  formative  evaluations 

4 

0.67 

Conduct  summative  evaluations 

3 

0.50 

Manage  instructional/training  projects 

4 

0.67 

Materials 

Materials  consisted  of  three  parts:  (i)  A background  survey;  (ii)  A worked  out  example  including  a sample 
problem  scenario  and  a sample  solution  for  that  scenario;  (iii)  Problem  scenario  to  be  worked  out  by  participants. 

The  problem  scenario  (see  Figure  1)  was  constmcted  in  such  a way  that  it  resembled  an  authentic 
instructional  design  problem.  It  included  design  and  development  of  instructional  materials  on  the  topic  of 
integrating  technology  into  middle  school  science  teaching  to  be  delivered  via  the  Internet  to  rural  teachers  all  across 
the  USA  within  established  project  specifications. 
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Procedure 

The  study  was  conducted  in  two  consecutive  sessions.  The  first  session  was  an  informational  session. 
Biographical  information  of  participants  was  collected  via  background  survey.  Then,  a brief  description  of  the 
second  session  was  presented. 

During  the  second  session,  the  participants  were  provided  with  an  example  explaining  the  task  they  were 
about  to  undertake.  Then,  they  were  given  the  problem  scenario  and  were  asked  to:  (i)  take  some  time  to 
individually  reflect  on  the  problem  scenario  and  provide  their  assumptions  and  contextual  remarks;  (ii)  create  a 
representation  of  the  given  problem  scenario  identifying  the  key  factors  (such  as  entities,  rates,  or  processes)  and  the 
relationships  between  the  key  factors;  (iii)  annotate  each  node  (factor)  and  each  link  (relationship)  in  their 
representation  by  further  elaborating  on  the  details;  (iv)  provide  recommendations  for  the  solution  of  the  given 
problem  scenario  based  on  their  analysis  as  reflected  in  their  representations. 

Protocol  analysis  was  used  to  analyze  participants’  annotated  representations  of  the  given  ID  problem 
scenario,  their  assumptions  and  their  recommendations  for  solution.  Three  main  sources  of  information  were  used 
for  the  development  of  the  coding  scheme:  (i)  instructional  design  elements,  which  emerged  from  the  initial 
qualitative  analysis  of  participant  data;  (ii)  established  elements  and  activities  in  instructional  design  models  (for  a 
review  of  instructional  design  models,  see  Gustafson  & Branch,  1997);  and,  (iii)  general  methods  and  guidelines 
suggested  by  Ericsson  & Simon  (1984)  to  develop  a coding  scheme  for  protocol  analysis.  Through  frequency 
counts,  the  researcher  sought  for  a recognizably  similar  pattern  between  the  expert  representations. 


You  are  the  manager  of  a grout)  of  instructional  develoners  and  educational  technologists  at  a small 

private  enterprise  named  Learnins  Designs  for  Fun  and  Profit.  Ltd.  located  in  the  mid-western  United 

States.  Your  team  is  working  on  a project  called  “Six  Simple  Steps”  that  involves  the  development  of 

a set  of  six  short  learning  modules  on  the  topic  of  integrating  technologv  into  middle  school  science 

teaching  to  be  delivered  via  the  Internet  to  rural  teachers  all  across  the  USA.  The  project  team 

includes  a manager  Ivoul.  an  instructional  designer,  a media  and  graphics  specialist,  a web  specialist. 

a subject  matter  specialist,  and  a consulting  middle  school  science  teacher.  The  project  is  currentlv  at 

the  end  of  the  eighth  month  of  a twelve-month  effort.  There  is  a bonus  of  10%  for  on-time  completion 

according  to  established  reauirements.  There  is  no  provision  for  cost  overruns  other  than 

renegotiating  the  contract,  which  is  with  a large  book  publisher. 

The  completed  set  of  modules  is  supposed  to  be  readv  for  a one-week  field  test  at  selected  schools  in 

six  states  in  two  weeks  time.  Four  of  the  six  modules  have  been  completed  and  have  been  tested 

informallv  within  vour  enterpris  e.  Those  four  are  believed  to  be  readv  for  the  field  test.  Work  has  not 

vet  begun  on  the  remaining  two  modules:  (a)  Supporting  Web-based  Collaborative  Experiments  in 

Earth  Science,  and.  (bl  Conducting  Webauests  for  Science  Education.  The  framework  and  approach 

used  in  the  previous  four  modules  are  somewhat  adaptable  and  suitable  for  the  remaining  two 

modules.  Based  on  the  time  and  expertise  used  to  develop  the  first  four  modules,  it  appears  verv 

unlikelv  that  the  team  can  complete  the  remaining  two  modules  and  conduct  an  internal  test  within 

two  weeks.  In  order  to  resolve  this  situation,  vour  task  is  to  first  determine  what  factors  are  relevant 

to  successful  completion  of  the  effort.  Based  on  that  analvsis.  vou  will  make  a recommendation  to  the 

President  of  the  companv  with  regard  to  resource  allocation  to  support  the  completion  of  the  effort. 

Figure  1.  Instructional  design  problem  scenario. 


Results 

The  analysis  of  the  protocols  has  consisted  of  two  steps,  namely:  the  examination  of  participant  protocols  in 
search  of  overall  patterns  and  the  coding  of  events  related  to  those  patterns. 

The  qualitative  analysis  was  guided  by  the  generation  of  initial  research  hypothesis  to  further  elaborate  on 
the  similarities  between  the  expert  responses  to  the  given  instructional  design  problem.  These  questions  evolved 
through  the  initial  analysis  of  the  protocols.  The  questions  addressed  how  expert  instructional  designers  approached 
complex  design  problem  solving  task  and  whether  it  was  possible  to  graphically  represent  the  common  elements  in 
their  approach. 

Table  2 presents  the  frequency  distribution  of  resulting  coding  categories.  From  the  ID  process  perspective, 
it  was  evident  that  expert  instructional  designers  viewed  the  instructional  design  process  in  its  entirety,  including  all 
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of  the  design  elements  (i.e.,  instructional  analysis,  design,  development,  evaluation,  and  feedback)  that  are  addressed 
in  a generic  ID  model. 

A number  of  common  themes  emerged  from  the  expert  protocols.  These  confirmed  the  findings  of  the 
previous  studies  (Goel  Sl  Pirolli,  1989;  Peiez  et  ah,  1995;  Rowland,  1992)  on  the  characteristics  of  instructional 
design  experts.  Experts  tried  to  get  a holistic  picture  of  the  problem  situation,  viewed  the  problem  in  its  entirety,  and 
then  they  focused  on  one  component  of  the  problem,  generated  an  initial  design  and  evaluated  it.  Based  on  the 
results  of  evaluation,  they  accepted,  rejected,  or  modified  it.  The  expert  designers  then  extended  the  particular 
solution  to  other  areas  of  the  “whole  design’’  and  evaluated  it.  From  the  responses  of  the  participants,  it  was  evident 
in  this  study  that  expert  instructional  designers  viewed  the  instructional  design  process  in  its  entirety,  including  all  of 
the  design  elements  (i.e.,  instructional  analysis,  design,  development,  evaluation,  and  feedback)  that  are  addressed  in 
a generic  ID  model. 


Table  2.  Coding  categories  and  their  frequency  counts 


Coding  Categories 

Frequency 

(N=6) 

Percent 

1.  Instructional  analysis  (includes  determining  goals  and  objective; 
task  analysis;  target  group  analysis;  etc.) 

2 

0.33 

2.  Design  (includes  sequencing,  determining  modules,  duration  of 
instruction,  construction  of  design  templates,  test  items,  etc.) 

6 

1.00 

3.  Develop  module  (includes  instructional  materials  and  media) 

6 

1.00 

4.  Pilot  testing  (includes  formative  evaluation  of  instructional 
materials  developed,  analysis  of  the  pilot  test  results,  and  feeding 
the  results  of  pilot  test  to  improve  design  template) 

5 

1.00 

5.  Client  (includes  requirements,  constraints,  flexibility,  budget, 
deadlines,  satisfaction,  etc.) 

3 

0.50 

6.  President  requirements  (includes  requirements,  constraints, 
commitments,  and  flexibility) 

3 

0.50 

7.  Project  budget  (estimated  budget  vs.  actual  costs) 

4 

0.67 

8.  Personnel- requirements,  expertise,  availability  (design  & 
development  team,  SME,  pilot  test  group,  etc.) 

5 

0.83 

9.  Delivery  schedule  (estimated  vs.  actual) 

5 

0.83 

10.  Overall  design  & development  time  (estimated  vs.  actual) 

5 

0.83 

11.  Outsourcing 

2 

0.33 

12.  Team  expertise 

3 

0.50 

13.  Quality  of  materials  developed 

3 

0.50 

Expert  Causal  Influence  Diagram 

Guided  by  the  research  question,  a second  level  of  analysis  aimed  at  constructing  a causal  influence 
diagram  (CID)  based  on  the  similarities  of  the  expert  representations.  This  stage  involved  both  the  analysis  of  the 
nodes  (factors)  and  the  relationship  between  these  nodes  (links)  across  expert  representations.  Annotations  of  nodes 
and  links  were  used  to  clarify  the  meaning  attached  to  these  elements  by  the  experts. 

Causal  influence  diagram  is  a commonly  used  mapping  technique  by  system  dynamics  community  to 
represent  the  casual  relationships  between  the  system  components  (Ford  & Sterman,  1998;  Vennix,  Anderson, 
Richardson,  & Rohrbaugh,  1994;  Vennix  & Gubbels,  1994).  It  can  be  viewed  as  a kind  of  concept  mapping 
technique  tailored  especially  for  dynamic  and  complex  domains.  It  visually  represents  the  dynamic  influences  and 
interrelationships  that  exist  among  a collection  of  variables.  A causal  influence  diagram  consists  of  a collection  of 
nodes  (key  factors)  and  links.  The  nodes  maybe  entities  that  can  be  counted,  such  as  products  or  people,  or  they 
might  be  rates  or  processes  that  represent  how  things  are  changing.  An  annotated  causal  diagram  has  a description 
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associated  with  each  node  to  indicate  the  specific  nature  of  that  factor.  The  links  connecting  nodes  are  directional  to 
show  how  one  conceives  of  the  causal  relationship  among  the  factors  (e.g.,  a rate  of  change  in  weight  might  have  a 
causal  influence  on  blood  pressure-  the  arrow  in  this  case  would  point  from  rate-of-weight-change  to  rate-of-blood 
pressure-change).  The  links  are  also  annotated  as  either  -i-  (SAME)  or-  (OPPOSITE)  to  indicate  whether  a change 
in  the  causal  factor  tends  to  create  a change  in  the  same  or  opposite  direction  in  the  affected  factor. 


Eigure  2.  Casual  influence  diagram  developed  from  the  similar  patterns  in  expert  responses 

Eigure  2 presents  the  causal  influence  diagram  developed  based  on  the  similar  patterns  between 
representations  of  expert  instructional  designers.  In  a way,  this  diagram  represents  the  underlying  model  that  expert 
instructional  designers  utilize  while  they  are  dealing  with  similar  instructional  design  problems. 

According  to  this  expert  causal  influence  diagram  (Eigure  2),  expert’s  mental  model  of  the  key  factors  and 
relationships  of  these  key  factors  in  the  given  problem  case  can  be  described  as  follows:  Client’s  requirements, 
determines  the  project  size,  scope,  budget,  and  timelines.  Increased  client  requirements  mean  increased  pressure  on 
the  personnel  to  meet  client  requirements.  Initial  project  specifications  state  the  number  of  personnel  that  can  be 
hired  for  the  project.  As  the  size  of  the  project  increases,  so  does  the  required  number  of  personnel.  If  more 
personnel  are  hired  then  it  will  be  possible  to  more  quickly  design  and  develop  the  materials.  Therefore,  one  of  the 
recommendations  from  experts  was  to  try  to  re -negotiate  with  the  client  & the  president  of  the  design  company  to 
hire  additional  staff  or  to  outsource  some  of  the  design  work. 

More  modules  mean  more  pilot  testing.  Pilot  testing  insures  the  quality  of  the  materials  produced,  but  it 
also  slows  down  the  delivery  of  the  materials.  However,  pilot  testing  also  contributes  to  the  team  expertise.  The 
more  pilot  testing  is  conducted,  the  more  knowledge  and  experience  they  will  have  about  what  works,  what  doesn’t 
work,  and  how  to  design  the  materials  better  and  quicker.  If  team  expertise  increases,  the  need  for  more  personnel 
decreases  because  existing  personnel  will  work  faster  and  better.  More  materials  delivered  means  more  client 
satisfaction.  Another  way  to  increase  client  satisfaction  is  to  increase  the  quality  of  materials  delivered.  As  client 
satisfaction  increases  there  will  be  more  orders  for  the  instructional  design  company.  In  turn,  pressure  of  the 
president  on  the  personnel  will  decrease. 

Expert  Recommendations  for  Solution 

As  far  as  the  recommendations  for  solution  are  concerned,  a number  of  common  themes  emerged  from  the 
expert  protocols.  Eirst,  experts  wanted  to  examine  the  whole  production  process,  including  resources  (personnel, 
budget  estimates,  etc.),  task  division,  and  production  schedule  to  determine  causes  of  the  problem.  The  second 
common  expert  reaction  was  to  analyze  the  status  of  remaining  resources  and  other  production  variables.  Based  on 
these  two  sources  of  input,  experts  typically  focused  on  the  main  piece  of  the  problem  (according  to  their  individual 
judgments),  recommended  a solution,  and  extended  their  solutions  to  include  other  pieces  of  the  overall  problem. 
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Based  on  their  earlier  experiences,  some  ejperts  provided  a number  of  rule-of-thumbs  (heuristics)  that 
could  be  applied  to  the  given  problem  scenario.  After  the  early  analysis  of  the  problem,  two  main  approaches  were 
observed.  Some  of  the  experts  wanted  to  report  this  to  president  and  to  the  client  for  a possible  renegotiation  before 
revising  the  project  management  plan.  Others  did  not  consider  this  as  an  alternative.  They  preferred  to  revise  the 
overall  project  management  piece  to  ensure  on  time  delivery.  In  order  to  push  the  production  forward,  a number  of 
alternatives  were  proposed.  The  most  common  ones  included  construction  of  design  templates  based  on  the  pilot 
testing  data  of  earlier  modules,  reusability  of  previously  developed  materials,  and  outsourcing. 

Team  expertise  also  emerged  as  an  imp  ortant  factor  in  the  development  process.  Experts  argued  that  after 
the  development  and  pilot  testing  of  the  first  four  modules,  the  ID  team  would  have  the  expertise  to  more  quickly 
finish  the  remaining  modules.  The  feedback  from  the  pilot  testing  from  the  first  four  modules  could  be  used  to 
improve  the  design  template.  By  this  way,  it  would  be  possible  to  use  this  template  to  develop  the  rest  of  the 
modules.  If  time  is  of  concern,  experts  suggested  not  to  pilot  test  the  remaining  modules.  They  agreed  on  that  it 
would  be  a compromise  the  quality  of  the  materials,  however,  they  argued  that  since  previous  materials  were  pilot 
tested  and  the  design  template  was  improved  based  on  the  results  of  the  pilot  testing  it  would  not  be  a big 
compromise  as  far  as  the  quality  was  concerned. 


Discussion 

The  goal  of  instruction  is  the  facilitation  of  learning  (Gagne  & Merrill,  1990;  Scandura,  1995;  Spector  & 
Anderson,  2000;  Tennyson  & Morrison,  2000).  Now,  we  are  fairly  successful  in  facilitating  learning  in  well- 
structured,  relatively  simple  domains.  However,  majority  of  real-life  situations  are  not  well-structured.  Humans 
often  face  complex,  ill-structured  problem  solving  situations  while  performing.  Unfortunately,  an  important  yet  little 
understood  question  remains  as  how  to  facilitate  learning  and  promote  expertise  in  such  complex,  ill -structured 
domains  (Spector  & Anderson,  2000;  Sterman,  1994) 

Spector  et  al.  (in  press)  propose  an  investigative  design  framework  for  facilitating  learning  in  complex 
domains.  It  includes  specific  instructional  methods  that  can  be  manipulated  in  controlled  studies  to  determine  which 
method  works  best  in  facilitating  learning  in  complex  domains. 

An  assessment  methodology  based  on  causal  influence  diagrams  is  at  the  heart  of  this  design  framework.  Causal 
influence  diagrams  are  commonly  used  in  the  field  of  system  dynamics  to  model  a complex,  dynamic  system  in  a 
way  that  helps  to  understand  and  analyzes  their  underlying  mechanisms  and  structures  (Laukkanen,  1998).  Causal 
influence  diagrams  have  been  investigated  (Christensen  et  al.,  2000;  Spector  & Davidsen,  1997a,  1997b)  as  an 
assessment  tool  in  a variety  of  domains.  These  studies  aimed  at  determining  whether  or  not  expert  patterns  of 
thinking  about  given  problem  scenarios  in  complex  domains  existed;  whether  novice  patterns  were  recognizably 
different;  and  whether  or  not  improvements  in  learning  after  instruction  could  be  assessed  using  this  methodology 
(Christensen  et  al.,  2000).  The  basic  answers  to  these  questions  were  positive  (Spector  et  al.,  in  press). 

This  study  could  be  considered  as  a preliminary  attempt  to  validate  the  results  of  previous  studies  in  the 
complex  domain  of  instructional  design.  However,  there  are  a few  differences  between  the  procedures  involved  in 
this  study  and  in  the  previous  studies.  First  of  all,  in  this  study,  expert  instructional  designers  were  not  directly  asked 
to  create  a causal  influence  diagram  representing  the  given  problem  scenario.  This  was  done  to  avoid  any  possible 
confusion  that  could  have  aroused  from  lack  of  background  knowledge  with  that  specific  mapping  technique  (in 
previous  studies  participants  had  been  provided  with  brief  a introductory  workshop  on  system  dynamics  -including 
CID-  or  they  already  possessed  the  related  background).  Instead  of  directly  being  asked  to  create  a CID,  the 
participants  in  this  study  were  asked  to  create  a representation  of  the  given  problem  scenario,  which  includes  the 
key  factors  (entities,  processes,  or  rates)  in  the  problem  and  the  relationships  between  these  key  factors  (links).  They 
were  also  instructed  to  annotate  each  node  (factor)  and  link  (relationship)  in  their  representations  in  order  to  prevent 
any  misinterpretation  during  analysis  and  to  have  a richer  set  of  data  with  respect  to  their  representations. 
Furthermore,  participants  were  provided  with  an  example  problem  scenario,  an  example  solution  for  that  scenario, 
and  an  explanation.  The  exa  mple  solution  included  a very  simplified  version  of  a causal  influence  diagram  but  it  was 
called  a representation . 

Secondly,  in  this  study  the  participants  were  asked  to  provide  their  assumptions  and  contextual  remarks 
before  creating  their  representations.  This  allowed  the  researcher  to  better  understand  how  these  experts  interpreted 
the  problem  scenario  as  well  as  their  approach  to  solving  a complex  instructional  design  problem.  The  data  provided 
from  this  section  proved  to  be  very  helpful  during  analysis.  It  did  not  only  include  assumptions  and  contextual 
remarks  but  also  some  initial  reactions  to  the  problem  scenario,  a number  of  heuristics,  rules -of-thumbs,  related 
previous  experiences  as  well.  It  was  evident  that  this  additional  part  helped  participants  to  reflect  on  the  problem 
scenario  based  on  their  previous  experiences  before  actually  proceeding  with  the  representation. 
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The  third  difference  between  this  and  previous  studies  was  the  recommendation  section.  After  participants 
created  and  annotated  their  representations,  they  were  asked  to  provide  their  recommendations  for  the  given  problem 
scenario,  based  on  their  representations.  This  data  gathered  from  this  section  was  helpful  in  understanding  what 
experts  would  do  in  similar  problem  situations.  The  contribution  of  the  data  gathered  from  this  section  was  also 
indispensable  to  further  understand  how  experts  approach  problem  solve  in  the  domain  of  instructional  design. 

This  final  remark  concerns  the  expert  representations  gathered  between  this  and  the  previous  similar  studies  (i.e., 
Christensen  et  ah,  2000;  Spector  & Davidsen,  1997a,  1997b)  In  our  study,  experts  were  highly  creative  as  far  as  the 
way  they  created  their  representations  were  concerned.  Since  they  were  not  directly  instructed  to  create  causal 
influence  diagrams,  their  representations  looked  somewhat  different  than  what  a CID  would  look  like. 

Some  experts  came  up  with  very  rich,  detailed  but  visually  somewhat  different-looking  representations.  A 
close  examination  on  the  characteristics  of  these  experts  showed  that  as  the  years  of  experience  increased,  generally, 
the  level  of  detail  and  the  creativity  put  into  the  representation  also  increased.  This  is  not  surprising  given  the 
previous  research  on  expertise  (Chi  et  ah,  1988).  But  there  were  a few  outliers  to  this  rule.  For  instance,  one  of  the 
experts  who  had  20  years  of  experience  in  instructional  design  provided  much  more  comprehensive,  very  detailed 
representation  than  the  one  with  34  years  of  active,  hands-on  experience.  With  the  help  of  the  background  survey,  it 
was  possible  to  provide  an  explanation  for  such  situations.  The  expert  who  had  20  years  of  experience  stated  that  he 
regularly  managed  instructional/training  design  projects  in  his  current  job  besides  designing  and  developing 
instructional  materials.  On  the  other  hand,  the  expert  with  34  years  of  experience  only  actively  performed  tasks 
related  to  the  design  and  development  of  instructional  materials.  Since,  the  problem  scenario  included  managing 
instructional  design  resources  this  difference  in  the  type  of  experiences  seemed  reasonable  to  explain  for  the  rather 
less  experienced  designers  creating  a more  comprehensive  representation  than  the  more  experienced  one.  In  other 
words,  since  experts  draw  from  their  previous  experiences,  the  type  of  design  tasks  they  are  regularly  involved 
becomes  another  very  important  factor  in  how  they  approach  particular  design  problems. 

Despite  the  differences  between  the  procedures  of  this  and  previous  similar  studies,  experts’  representations 
in  this  study  exhibited  recognizably  similar  patterns.  Furthermore,  from  these  similar  patterns  it  was  possible  to 
create  a causal  influence  diagram  that  represents  a common  expert  view  of  the  given  problem  scenario  (see  Figure  1, 
p.  5).  Given  these  promising  results  of  this  exploratory  study,  a follow-up  study  is  planned  with  larger  number  of 
experts  and  novices  with  the  aim  of  validating  the  results  of  this  study  and  related  previous  studies  in  the  domain  of 
instructional  design.  If  it  can  be  concluded  that  expert  instructional  designers  create  recognizably  similar  causal 
representations  of  common  problem  scenarios  and  the  representations  developed  by  novices  are  significantly 
different  than  those  of  experts,  are  rather  chaotic,  and  are  not  comprehensive  then  expert  representations  can  provide 
a benchmark  for  learner  progress  by  comparing  how  learner  representations  change  through  instruction  to  resemble 
or  fail  to  resemble  expert  patterns.  As  a result  we  can  test  the  effectiveness  of  different  instructional  approaches  in 
facilitating  learning,  and  thus,  we  can  contribute  to  understanding  the  development  of  and  facilitating  the  promotion 
of  expertise  in  complex  domains. 
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Abstract 

Although  an  invigorating  approach  to  IT  theory  and  research,  the  postmodern  agenda  would  benefit  from 
clearer  articulation  and  further  refinement  of  ontological,  epistemological,  and  methodological  positions. 
Consequently,  I reveal  possible  weaknesses  in  the  position  and,  in  the  spirit  of  scholarly  dialogue,  counter  with  a 
critical-realist  perspective  that  presents  a potentially  more  innovative  and  defensible  approach  to  the  discovery  of 
scientific  knowledge.  In  the  end,  I concur  with  the  postmodernists  that  there  are  issues  such  as  race,  gender  and 
ethnicity  that  must  be  addressed  in  IT.  My  point  of  difference  is  that  as  a scientific  venture,  we  must  be  more  public 
and  transparent  in  our  discourse  for  change  to  take  place. 


Introduction 

For  little  over  a decade  a steady  stream  of  work  that  promotes  a postmodern  posture  has  infiltrated 
instructional  technology  (IT)  theory  and  research  (Bryson  & De  Castell,  1994;  Conlon,  2000;  Hlynka,  1991;  Lawson 
& Comber,  2000;  Lien,  Jeng,  & Yin,  1998;  Sherman,  2000;  Solomon,  2001;  Yeaman,  Hlynka,  Anderson,  Damarin, 

& Muffoletto,  1 996).  Although  there  certainly  are  thought-provoking  contributions  and  benefits  provided  by  this 
provocative  perspective,  my  sense  is  that  our  postmodernist  colleagues  continue  to  struggle  to  clarify  their  views. 
Accordingly,  I request  greater  clarity  along  three  stances:  1)  the  rejection  of  positivistic,  rationalist  science,  2)  the 
acceptance  of  a (radically)  constructivistic  epistemology,  and  3)  the  reliance  upon  a singular  method  of  inquiry.  My 
aim  is  to  gently  ‘push  back’  so  as  to  further  scientific  development  in  our  field.  Additionally,  I offer  my  own  view — 
a critical-realist  perspective — that  may  permit  for  further  progress  in  the  discourse.  Before  initiating  this  dialogue, 
though,  it  is  necessary  to  clarify  what  I mean  by  the  postmodern  agenda  in  IT. 

Extrapolating  from  Solomon’s  (2000,  p.  8)  account  postmodernism  can  be  conceptualized  as  an  intellectual 
movement,  a condition,  or  a philosophy  that  claims  certain  values  and  assumptions  (Bailey,  1999;  Bryson  & De 
Castell,  1994;  De  Vaney,  1998;Yeaman,  et  ah,  1996).  As  an  intellectual  movement,  postmodernists  have  argued  for, 
in  place  of  science  and  rationality,  a focus  on  the  deconstruction  of  metanarratives  to  reveal  inherent  gender,  class, 
race,  ethnic,  and  sexual  inequalities  e)tant  in  a given  discipline  or  domain  of  inquiry  (Bailey,  1999;  De  Vaney, 

1998;  Sherman,  2000).  As  a condition,  postmodernism  is  a reaction  to  the  values  of  ‘modernity’  including  rational 
science,  uninterrupted  progress,  and  deterministic  technology.  Two  outcomes  of  this  condition  that  directly  impact 
education  are  that  students  have  become  subjugated  to  rational  science  and  learning  has  been  commodified  (Conlon, 
2000;  De  Vaney,  1998;  Sherman,  2000).  Finally,  postmodernism  can  be  seen  as  a philosophy,  the  perspective  that 
informs  our  present  discussion.  As  a philosophy,  postmodernism  promotes  particular  assumptions  linked  with  the 
movement  and  condition  as  follows:  (a) pluralism:  there  is  no  dominant  worldview  and  all  should  be  allowed  to 
coexist,  (b)  eclecticism:  ideas  and  methods  should  be  applied  in  the  manner  of  the  bricoleur,  (c)  truth:  a relativistic 
stance  that  gives  precedent  to  subjective  experience;  (d)  knowledge:  a rejection  of  universal  metanarratives  in  favor 
of  constructivism;  (e)  language:  the  primary  mediator  between  individuals  and  society;  (f)  complexity:  rejection  of 
deterministic  predictability  in  favor  of  chaos;  and  (g)  self,  a multi-dimensional  entity  in  constant  struggle  with  the 
previous  assumptions  (Sherman,  2000,  p.52;  Solomon,  2000,  p.  12-15).  As  I will  illustrate  below,  these  assumptions 
underlie  the  three  prominent  stances  of  the  postmodern  agenda  toward  theory  and  research  in  IT. 

Thus,  my  presentation  runs  as  follows.  First,  I interpret  and  then  critique  three  stances  within  the 
postmodern  agenda  regarding  rational  science,  a constructivistic  epistemology,  and  deconstruction  as  a mode  of 
inquiry.  The  responses  I forward  are  that  the  postmodern  stance  on  science  is  outdated,  the  epistemology 
questionably  defensible,  and  the  inquiry  insufficient.  Keeping  with  my  primary  aim  of  promoting  critical  dialogue  I 
conclude  by  presenting  what  I interpret  as  a contributory  refinement — a proposal  for  a critical-realist  perspective  on 
IT  theory  and  research. 
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A Critique  of  Three  Stances  within  the  Postmodern  Agenda 

My  intent  in  this  section  is  to  engage  the  advocates  of  postmodernism  in  what  can  be  viewed  as  a 
continuing  critical,  yet  overall  healthy,  dialogue  (cf..  Reeves,  2000;  Reigeluth,  1997;  Schwen,  2001;  Wilson,  1997). 
It  is  in  this  spirit  that  I have  enclosed  perspectives  and  references  that  support  the  postmodern  agenda  (see  Appendix 
A). 

The  Rejection  of  Positivistic,  Rationalist  Science 

From  my  interpretation,  advocates  of  the  postmodern  agenda  resonate  most  clearly  with  the  critique  of 
conventional  positivistic,  rationalistic  science  in  IT  (De  Vaney,  1998;  Solomon,  2000;  Wilson,  1997;  Yeaman,  et  ah, 
1996).  This  stance  is  supported,  in  part,  by  assumptions  concerning  truth,  knowledge,  and  complexity.  The 
postmodern  objection  is  that  conventional  research  and  development  in  IT,  driven  by  its  behavioral  science  roots, 
has  been  concerned  primarily  with  “the  use  of  precision-based  methods,  measurement,  replicability,  predictability 
and  order”  (Solomon,  2000,  p.  5).  This  stance  is  understandable  given  that  a primary  source  of  the  postmodernism 
perspective  is  critical  theory  (Alvesson  & Deetz,  1996;  Solomon,  2000,  2002;  Yeaman,  et  al.  1996).  Briefly,  critical 
theory  maintains  that  since  science  is  an  inherently  social  process,  in  which  egos  and  ideologies  frequently 
overcome  rational  thought,  it  must  be  riddled  with  nontheoretical  interests.  For  instance,  in  their  critical  examination 
of  the  ‘technicist  view’  of  educational  technologists,  Bryson  and  De  Castell  (1994,  p.  207-210)  conclude  that 
existing  social  norms  and  relations  of  power  between  researchers  and  students  were  cloaked  in  a discourse  of  value- 
neutral  technology,  the  discourse  itself  skewed  by  modernist  visions  of  progress  held  by  the  researchers. 
Consequently,  postmodernists  question  whether  rational  science  alone  is  sufficient  to  offer  the  best  path  to  scientific 
knowledge  in  IT  (Solomon,  2000,  2002;  Wilson,  1997;  Yeaman  et  al.,  1996). 

My  response  to  this  stance  is  that  postmodernists  have  taken  an  extreme  opposition  on  a view  of  rational 
science  that  may  itself  be  anachronistic.  That  is,  the  idea  of  science  as  a practice  of  uncovering  some  ultimate  reality 
in  search  of  a final  truth  is  so  outdated  as  to  be  a caricature  (Willower,  2001).  Moreover,  despite  what  the 
postmodernists  may  claim  or  imply,  this  critical  position  on  science  is  historically  far  from  unique.  On  the  contrary, 
philosophers  of  science  and  social  scientists  have  debated  this  issue  from  early  in  the  last  century  (Bernstein,  1976; 
Bhaskar,  1975;  Court,  1999;  Kuhn,  1962;  Popper,  1962).  As  a way  to  refresh  the  dialogue,  then,  what  I offer  below 
is  a more  innovative  mind-set.  My  contention  is  that  protocols  derived  from  a critical-realist  perspective  may  allow 
us  to  address  more  adequately  the  tensions  of  domination  and  emancipation  so  oft -cited  by  postmodernists  (Bryson 
& De  Castell,  1994;  De  Vaney,  1998;  Sherman,  2000;  Yeaman,  2000;  Yeaman  et  al.,  1996).  The  point  here  is  not 
that  the  postmodernist  agenda  has  abandoned  rigorous  empirical  methods,  e.g.,  ethnography,  action  research,  and 
narrative  inquiry  (although  I will  argue  that  they  rarely  fully  employ  these  methods).  The  issue  is  that  they  have  yet 
to  offer  an  alternative  to  the  logic  of  scientific  discovery,  by  which  the  use  of  methods  is  governed  (Corson,  1999; 
Popper,  1962;  Swann,  1999;  Thornton,  2000).  Thus,  the  clarity  I am  seeking  here  requires  that  postmodern 
advocates  better  articulate  the  alternative  logic  of  discovery  of  knowledge  they  propose  in  place  of  science. 

An  Acceptance  of  a (Radically)  Constructivistic  Epistemology 

Drawing  from  their  assumptions  on  the  importance  of  pluralism  and  language,  and  the  notions  of  truth  and 
self,  the  intent  of  the  postmodern  agenda  is  to  replace  monolithic,  grand  narratives  contrived  by  scientists  with 
multiple,  alternative  perspectives  that  are  permitted  to  coexist  (Bryson  & De  Castell,  1994;  Lawson  & Comber, 
2000;  Solomon,  2000,  2002;  Yeaman,  et  al.  1996).  This  notion  of  the  possibility  of  subjective  interpretation  is 
clearly  a byproduct  of  the  relationship  postmodernism  has  to  postructuralism.  From  the  poststructuralist  viewpoint, 
there  is  no  authoritative,  foundational  truth  that  we  can  come  to  know.  On  the  contrary,  each  individual  constructs 
his  or  her  own  knowledge  of  the  world  and  thus  any  one  construction  should  be  permitted  to  coexist  with  any  other 
(Solomon,  2000,  2002;  Yeaman,  et  al.,  1996).  From  one  instructional  technologist’s  perspective  (Hay,  1994,  p.23), 
this  situation  has  resulted  in  three  crises  in  educational  theory  and  practice.  The  first  crisis  in  representation 
challenges  the  conventional,  foundationalist  view  on  truth.  As  a result,  what  we  consider  as  knowledge  and  what  we 
convey  to  practitioners  or  teach  our  students  is  called  into  question.  The  second  crisis,  related  to  the  first,  deals  with 
authority.  If  knowledge  and  truth  are  questioned,  on  what  basis  can  we  justify  our  position  as  scientists  and 
practitioners?  Finally,  the  third  crisis  calls  into  question  the  status  of  the  self.  This  notion  of  an  extreme  subjectivity, 
where  the  individual  is  constmcted  from  discourse,  seriously  undermines  the  educator’s  justification  that  she  knows 
more  about  the  learner  than  the  learner  himself  does. 

In  this  reformulation  of  knowledge  and  its  scientific  discovery,  the  postmodernists  have  committed  two 
critical  errors.  First,  the  insightful  observation  that  science  is  not  conducted  by  rational  logic  alone  does  not  preclude 
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that  all  knowledge  is  relative.  The  issue  here  is  that  if  the  postmodern  position  on  epistemology  is  taken  to  an 
extreme,  then  no  account  can  be  made  for  intersubjective  understanding  among  scientists.  Although  individuals  can 
(and  do)  uniquely  construct  knowledge,  some  consensus  must  be  reached  for  scholarly  exchange  to  take  place  on  a 
collective  level.  As  Bailey  (1999)  has  cogently  stated  in  his  response  to  the  postmodern  attack  on  science,  “science 
is  a public  inter-subjective  activity...  and  thus  all  science,  however  flawed  its  individual  practitioners  may  be,  is 
ultimately  accountable’’  (p.  35).  Thus,  the  crises  in  educational  theory  and  practice  might  in  some  way  be  alleviated 
if  postmodernists  contemplate  that  social  norms,  such  as  those  imposed  by  professional  communities  of  scientists 
and  teachers,  enforce  regulative  accountability  of  knowledge  and  truth  (Wenger,  1998,  2000).  A second  error  has  to 
do  with  the  postmodernists  stumbling  over  their  own  attempts  at  breaking  down  myths  and  metanarratives.  If,  as 
they  claim,  ‘everything  is  subjective’  then  an  argument  cannot  be  made  for  the  preference  of  one  agenda  over 
another.  If  the  postmodernis  ts  claim  that  there  are  other  (or  ‘better’)  ways  of  coming  to  know  truth,  then  subjectivity 
has  been  defeated  (Bailey,  1999;  Dawson  & Prus,  1993,  1995). 

A Potential  Over -reliance  on  a Singular  Method  of  Inquiry 

Deconstruction  takes  as  one  of  its  major  tenets  that  “virtually  any  facet  of  cultural  life  can  be  interpreted  as 
a text  and  subsequently  deconstructed.’’  (De  Vaney,  1998,  p.  76;  Solomon,  2000,  p.  1 1).  This  stance  clearly  draws 
from  assumptions  regarding  pluralism,  truth,  language,  and  self.  Despite  the  postmodernist  resistance  to  algorithms, 
Yeaman,  et  al.  (1996,  p.260)  have  described  deconstruction  as  a process  for  revealing  inherent  dualities  that  are 
usually  associated  with  traditional  binaries,  e.g.,  good/bad,  global/local,  nature/technology,  normal/abnormal, 
male/female,  and  oppressor/oppressed.  In  many  instances,  the  methodology  follows  closely  that  of  discourse 
analysis  (Gee,  1999)  but  with  a particular  predilection  to  reveal  the  inadequacies  of  ‘modernity’.  The  essence  of  the 
methodology  is  based  on  the  belief  that  since  science  and  knowledge  are  primarily  mediated  by  language,  and 
foundations  upon  which  to  ground  any  one  perspective  no  longer  exist,  endless  interpretation  is  appropriate  and 
encouraged  (Solomon,  2000,  2002;  Yeaman,  et  al.  1996).  As  an  example  of  this  deconstructive  interpretation,  De 
Vaney  attempts  to  illustrate  how  contemporary  educational  technologists,  driven  by  modernist  prerogatives,  have 
ignored  gender,  race,  and  class  differences  to  construct  a subject  malleable  to  software  and  media  that  are  to  varying 
degrees  overtly  materialistic,  narcissistic,  sexist,  and  hedonistic  (1998,  p.77). 

Despite  the  relevant,  penetrating  insights  gained  from  this  alternative  methodology,  it  is  not  without 
weaknesses.  As  Reed  (2000,  pp. 525 -527)  has  indicated,  deconstruction  suffers  from  at  least  five  shortcomings:  1) 
constructivism:  as  noted  above,  reality  is  literally  ‘talked  and  texted’;  2)  nominalism:  conceptualizations  and 
explanations  are  nothing  more  than  ‘names’  or  ‘fictions’;  3)  determinism:  ironically,  human  agency  is  downplayed 
in  favor  determinate  rules  and  practices;  4)  localism:  micro-level  analyses  ignore  more  permanent,  hierarchical 
institutionalized  structures;  and  5)  reductionism:  ideologies  are  stripped  of  their  cultural  and  historical  context.  More 
serious  than  the  process  of  deconstruction,  is  the  incompleteness  of  the  overall  product.  That  is,  once  a 
deconstruction  has  taken  place  postmodern  advocates  seldom,  if  ever,  demonstrate  the  logical  next  move  of  action 
for  improvement  or  empowerment  (although  Bryson  and  De  Castell  (1994)  provide  an  exception).  Echoing  these 
sentiments.  Reeves  (2000,  p.24)  notes  that  although  benefits  can  be  gained  by  revealing  biases  in  educational 
innovations,  postmodern  research  is  potentially  sterile  when  comes  to  the  actual  improvement  of  conditions  for 
teaching  and  learning.  Thus,  what  I encourage  the  postmodern  contingent  to  consider  is  that  what  is  needed  is  not 
only  deconstruction,  but  also  construction,  for  example,  in  the  style  of  action,  development,  or  formative  research 
(Reeves,  2000;  Reigeluth,  1997). 

In  summary,  the  argument  I have  constructed  is  that  without  a socially  available  object  of  study  or  protocol 
for  determining  how  to  hold  scientists  accountable  for  their  knowledge,  the  postmodern  agenda  may  fail  to  bring  the 
laudable  advancements  or  improvements  they  seek.  My  question  to  postmodernists  is  that  if  everything  begins  and 
ends  with  interpretation,  then  how  are  we  to  perform  our  tasks  as  scientists,  informing  both  colleagues  and 
practitioners  of  ever  more  sophisticated  understandings  and  suggestions  for  innovation? 


Advancing  the  Dialogue — A Critical-Realist  Perspective  on  IT  Theory  and  Research 

What  I am  not  presenting  here  is  a definitive  argument,  but  merely  a tenable  counter-position  to  the 
postmodern  agenda  from  which  to  take  a critical  stance  toward  theoretical  and  research  practice  in  instructional  IT 
(Schwen,  2001).  As  I hope  to  illustrate,  however,  the  critical-realist  perspective  on  the  nature  of  the  subject  of 
inquiry  and  the  means  to  come  to  know  that  subject  are  an  improvement  on  postmodernist  views. 
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The  Social  Reality  of  Our  Subject  of  Inquiry 

The  critical-realist  perspective  takes  as  the  subject  of  inquiry  not  accepted  causal  social  structures  and 
mechanisms  (a  sort  of  naive  empiricism),  but  the  presuppositions  scientific  practice  has  about  these  ‘objective 
things’  (Corson,  1999).  Accordingly,  the  objects  of  our  study  should  be  the  theories  and  models  we,  as  a community 
of  scholars,  construct  of  social  structures  and  mechanisms  that  influence  learning  and  performance  (cf.,  Engestrom, 
1987;  Wenger,  1998).  Along  the  lines  of  Bhaskar  (1975),  the  proposition  offered  here  is  that  it  would  contribute 
greatly  to  scientific  development  in  IT  if  we  took  these  conceptualized  social  structures  and  mechanisms  (such  as 
communities  of  practice  or  activity  systems)  as  real.  Thus,  scientific  work  should  be  occupied  by  hypothesizing  the 
existence  of  social  structures  and  mechanisms,  and  then  to  detail  their  operation  (Corson,  1999). 

Although  similarities  may  be  found  with  postmodernism  in  this  perspective,  the  fundamental  difference  is 
ontological.  Whereas  postmodernists  would  permit  for  the  infinite  regress  of  interpretation  and  subsequent 
instability  of  reality,  the  critical-realist  would  counter  by  offering  that  for  open,  public  scientific  practice  subject  to 
refutation,  it  is  more  fruitful  to  assume  that  a socially  constructed  reality  exists  for  scientific  inquiry  (Reed,  2000; 
Tsoukas,  2000).  Another  way  to  say  this  is  that  postmodernist  analytical  and  methodological  work  tends  to  conflate 
agency  and  structure,  resulting  in  a ‘flat’  ontology  (Reed,  1997).  Michael  Reed  has  a nice  quote  that  captures  this 
preference  for  flux  and  transformation:  “The  world  of  the  actor-network  theorist,  as  that  of  the  ethnomethodologist 
[both  representative  of  the  postmodern  turn],  seems  to  consist  almost  totally  of  verbs  and  hardly  any  nouns;  there  is 
only  process,  and  structure  is  regarded  as  its  passing  effect’’  (1997,  p.  26).  Consequently,  what  I am  proposing  is  that 
the  critical-realist  perspective  provides  a workable  mind-set  for  scientists  in  IT.  Denying  the  tenability  of  this  mind- 
set could  potentially  undermine  attempts  to  address  the  metanarratives  hypothesized  by  the  postmodernist  scholar. 
For  conceptual  and  empirical  evidence  of  how  this  mind-set  might  be  brought  into  play,  the  reader  is  referred  to  the 
work  of  Cook  and  Yanow  (1993),  Engestrom  (1987),  Gherardi,  Nicolini,  and  Odella  (1998),  Henricksson  (2000), 
Wenger  (1998,  2000),  and  Yanow  (2000).  Each  of  these  pieces  posits  social  structures  and  mechanisms  (e.g., 
‘organizations  as  cultures’,  activity  systems,  communities  of  practice,  situated  curricula)  that  influence  learning  and 
performance  in  collective  settings.  Grounding  this  argument  more  clearly  in  IT  territory,  it  is  clear  that  a notion  of  a 
persistent  reality  gives  us  leverage  to  discuss  issues  of  race,  class,  and  gender  that  are  now  considered  critical  to 
responsible  development  of  knowledge  and  technology  in  our  field  (Voithofer  & Foley,  2002). 

A Critical  Posture  on  Knowing:  A Post-Popperian  Perspective 

From  a post-Popperian  perspective,  the  logic  of  the  growth  of  scientific  knowledge  is  based  upon  the 
premise  that  there  is  an  asymmetry  between  verification  and  falsification:  “While  no  number  of  true  singular 
observation  statements  can  verify  or  prove  the  truth  of  a universal  theory,  one  true  singular  observation  statement 
can  refute  if’  (Swann,  1999,  p.  22).  The  simple  logic  proposed  by  Popper  (1962;  Swann,  1999;  Thornton,  2000)  to 
refute  false  claims  can  be  represented  as  follows; 

PSi?  TT?  EE?  PS2 

In  this  formula,  PS  1 refers  to  a first  problem  statement  formulated  by  an  unexpected  encounter  with 
‘objective’  reality;  TT  refers  to  a tentative  theory  proposed  to  explain  that  problem;  EE  refers  to  the  process  of  error 
elimination  to  refute  any  portion  of  the  theory;  PS2  denotes  the  reformulated  understanding  and  thus  growth  of 
knowledge.  Thus,  a critical  posture  emerges  as  the  focus  here  is  not  on  attempting  to  establish  ‘truth’,  but  on  striving 
vigorously  to  remove  error  from  existing,  yet  admittedly  tentative,  knowledge. 

An  integral  part  of  this  process,  then,  is  the  formulation  of  bold  theories  that  subsequently  have  a high 
probability  of  falsifiability  (Swann,  1999;  Thornton,  2000).  This  is  accomplished  by  formulating  prohibitive 
hypotheses  that  contend  to  refute  a universal  statement  derived  from  a theory  by  identifying  limiting  conditions  to 
reveal  errors.  An  example  might  be  as  follows:  “There  does  not  exist  a situation  in  which  this  set  of  learning 
principles,  X,  when  adopted  in  practice  does  not  lead  to  greater  performance  with  regard  to  certain  objectives  or 
standards,  Y,  in  comparison  with  an  alternative,  Z’’  (adapted  from  Swann,  1999,  p.  28).  Thus,  instead  of  adhering  to 
a method  of  attempting  to  verify  the  ‘truth’  of  a particular  theory  (which  is  the  conventional  hypothetico-deductive 
approach  and  the  view  of  rational  science  that  postmodernists  appear  to  distrust),  the  critical-realist  stance  is  to 
search  vigorously  for  instances  were  a theory  has  a high  probability  of  being  falsified.  Examples  of  this  approach 
and  its  potential  contribution  to  educational  research  can  be  found  in  the  work  of  Corson  (1999),  Hillier  (1999),  and 
Swann  (1998). 

Another  way  of  labeling  this  position  is  to  say  that  it  is  “post-positivist”  (Philips  & Burbules,  2000).  As 
postpositivists  claim:  “[We]  . . .are  united  in  believing  that  human  knowledge  is  not  based  on  unchallengeable,  rock- 
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solid  foundations — it  is  conjectural  [emphasis  in  original].  We  have  grounds,  or  warrants,  for  asserting  the  beliefs, 
or  conjectures,  that  we  hold  as  scientists,  often  very  good  grounds,  but  these  grounds  are  not  indubitable.  Our 
warrants  for  accepting  these  things  can  be  withdrawn  in  the  light  of  further  investigation”  (Philips  & Burbules,  2000, 
p.  26).  This  statement  contains  (explicitly  and  implicitly)  three  critical  points  that  summarize  the  commitments  of  a 
postpositive  position.  The  first  is  that  knowledge  is  nonfoundational . What  this  means  is  that  the  postpositivist  does 
not  accept  the  tenets  of  empiricism  or  rationalism  that  claim  that  knowledge  is  founded  upon  either  experience  or 
rationality.  That  is,  neither  experience  nor  rationality  can  be  taken  as  the  foundation  upon  which  knowledge  can  be 
secured.  The  second  commitment  is  toward  a view  of  knowledge  as  conjectural . This  commitment  has  been 
explained  in  the  above  paragraphs.  The  third  (and  what  I deem  most  critical)  commitment  is  toward  seeking 
warrants  for  stated  assertions.  The  distinction  being  made  here  is  that  the  evidence  collected  to  support  any  claims 
cannot  be  based  on  authoritative  foundations  nor  can  it  be  said  to  be  “truth”.  Similar  to  the  burden  of  detectives  and 
lawyers  collecting  evidence  to  warrant  investigation  or  prosecution,  a scientist  taking  a post-positivistic  posture  must 
present  evidence  that  fulfills  domain -established  processes  by  which  claims  and  actions  are  authorized. 

In  the  end,  what  I am  trying  to  convey  in  this  admittedly  subtle  argument  are  two  things.  First, 
acknowledging  the  existence  of  an  ‘objective’  social  reality  will  permit  us  to  continue  to  conduct  inquiry  as  a 
community  of  scholars.  Second,  a post-Popperian  perspective  on  scientific  discovery  may  be  more  palpable  to 
postmodernists  as  it  provides  for  the  conjectural  nature  of  knowledge. 


Conclusion 

My  proposal  here  has  been  to  encourage  postmodernist  colleagues  to  continue  to  refine  their  ontological, 
epistemological  and  methodological  positions  as  well  as  overall  critique  of  conventional  scientific  inquiry.  In  fact, 
Voithofer  and  Foley  (2002)  in  response  to  Solomon’s  (2000)  original  article  have  begun  to  do  so  by  identifying  a list 
of  possible  priorities  for  research  within  the  postmodernist  agenda.  Three  of  these  priorities  entail  an  interest  in  how 
technology  shapes  pedagogy  and  curriculum  (p.  7),  a preference  for  multidisciplinary  and  interdisciplinary 
approaches  (p.  8),  and  a concern  with  language  and  meaning  (p.  8).  To  their  credit,  these  authors  are  striving  to  bring 
about  the  degree  of  clarity  in  the  postmodern  position  that  I ask  for  here. 

Certainly,  then,  critical  dialogue  is  needed  in  our  field.  Nevertheless,  for  the  critique  to  be  useful  it  must  be 
open,  public,  and  transparent  to  colleagues  and  practitioners  alike.  As  I have  attempted  to  argue  here,  a possible  way 
to  do  so  is  to  take  a ‘realist’  position  on  the  ontological  status  of  the  social  objects  under  investigation  and  a ‘critical’ 
position  as  to  what  we  can  know  about  those  objects.  By  these  means  we  may  better  be  able  to  attain  the  critical 
intersubjective  understanding  between  scholars  and  practitioners  that  advances  both  theory  and  practice.  I end  this 
piece  with  a quote  by  organizational  scientist,  Haridimos  Tsoukas,  that  echoes  my  sentiments: 

We  are  realists  simply  because  reality  is  where  is  (sic)  has  always  been,  outside  our  heads.  Insofar  as  we 
create  structures  through  patterns  of  sustained  interaction,  from  the  micro-level  of  the  small  group  right  to 
the  macrolevel  of  global  economic  systems,  we  are  confronted  by  real  structures  which  we  only  partially 
and  often  indirectly  and  unintentionally  have  helped  create.  Such  structures  cause  us  to  form  beliefs  about 
them.  In  turn,  our  descriptions  of  these  structures  (more  precisely,  how  we  describe  them),  are  matters 
which  depend  on  the  language-based  institutionalized  meanings  a community  of  actors  have  historically 
adopted.  It  is  processes  of  (history -shaped)  social  construction,  unfolding  in  time  and  space,  that  we,  as 
organizational  [and  IT]  researchers,  should  seek  to  study  (2000,p.  534). 
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Appendix  A 

Alternative  Perspectives  to  the  Critical-Realist  Approach 

Below  I have  listed  three  questions  that  are  pertinent  to  this  lively  discussion.  Associated  with  each 
question  are  selected  pieces  that  will  help  to  inform  the  reader  interested  in  taking  one  or  more  of  the  positions.  I 
look  forward  to  our  debate. 

1.  Has  it  not  already  been  well -articulated  that  rational  science  is  inadequate  for  instructional  technology  and  so 
the  postmodern  agenda  is  legitimate? 
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3.  Has  their  not  been  relevant  empirical  work  to  validate  the  contribution  of  the  postmodern  agenda  to 
instructional  technology  research? 
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Abstract 

This  paper  discusses  the  evaluation  of  a set  of  on-line  distance  learning  events  (brief  courses)  provided  to 
United  States  Department  of  Education  Middle  School  Drug  Prevention  and  School  Safety  Coordinators  (MSCs)  by 
the  National  Training  and  Technical  Assistance  Center  for  Middle  School  Coordinators.  It  includes  descriptive 
information  about  the  on-line  events,  and  some  of  the  guiding  questions  of  the  evaluation  design.  Attention  is  also 
given  to  the  importance  of  thinking  through  the  different  types  of  evaluation  data  that  can  be  gathered  about  on-line 
of  events,  and  the  potential  applications  of  this  information  with  illustrative  examples  from  data  collected  about  the 
MSC  events. 

Introduction 

The  United  States  Department  of  Education’ s (USED)  Safe  and  Drug-Eree  Schools  and  Communities 
Program  (SDESCP)  provides  funds  for  virtually  every  school  district  in  America  to  support  drug  and  violence 
prevention  programs.  In  1998,  USED  implemented  “principles  of  effectiveness”  that  required  all  SDESCP-funded 
programs  be  research-based.  To  help  schools  identify  and  adopt  research-based  drug  and  violence  prevention 
strategies,  the  1999  Safe  and  Drug-Eree  School  appropriation  of  $566  million  included  $35  million  for  the  first  year 
of  a new  Middle  School  Drug  Prevention  and  School  Safety  Coordinator  (MSC)  Initiative. 

The  initiative  supported  the  hiring  and  training  of  full-time  Middle  School  Coordinators  (MSCs)  in  each  of 
three  years  (1999,  2000,  2001).  Prom  1999  to  2001,  approximately  925  coordinators  were  hired.  MSCs  were 
funded  for  three  years  and  were  expected  to  do  such  things  as:  help  schools  identify  and  adopt  successful  research- 
based  drug  and  violence  prevention  strategies;  assess  school  crime  and  drug  problems;  and  work  with  parents, 
schools,  and  community  organizations  to  ensure  that  these  problems  were  addressed.  One  restriction  on  the 
initiative  was  that  the  funds  could  not  be  used  to  support  direct  delivery  of  prevention  services  by  MSCs. 

The  Safe  and  Drug-Eree  Schools  appropriation  also  included  funds  for  the  creation  of  a National  Training 
and  Technical  Assistance  Center  for  Middle  School  Coordinators.  Punded  by  the  USED  Office  of  Elementary  and 
Secondary  Education,  this  Center  was  operated  by  Health  and  Human  Development  Programs  (HHD),  a division  of 
Education  Development  Center,  Inc.  (EDC)  in  Newton,  Massachusetts.  Social  Science  Research  and  Evaluation, 

Inc.  (SSRE)  served  as  evaluator  for  the  National  Training  Center. 

This  paper  discusses  the  evaluation  of  a set  of  on-line  distance  learning  events  (brief  courses)  provided  to 
MSCs  by  the  National  Training  Center.  It  includes  descriptive  information  about  the  on-line  events,  and  some  of  the 
guiding  questions  of  the  evaluation  design.  Attention  is  also  given  to  the  importance  of  thinking  through  the 
different  types  of  evaluation  data  that  can  be  gathered  about  on-line  of  events,  and  the  potential  applications  of  this 
information  with  illustrative  examples  from  data  collected  about  the  MSC  events. 

Methods 

This  section  first  briefly  describes  two  different  types  of  trainings  provided  to  MSCs:  face-to-face  core 
training  workshops,  and  (2)  on-line  continuing  education  distance  learning  events.  It  then  describes  the  evaluation 
design,  testing  instruments,  and  data  collection  schedules  used  for  the  on-line  events. 
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Face-to-Face  Core  Training  Workshops 

Between  February  2000  and  April  2002,  nine  five-day  face-to-face  core  training  workshops  were  provided 
to  MSCs.  MSCs  were  required  to  attend  one  of  these  very  similar  workshops.  Each  consisted  of  both  large  group 
presentations  and  small  group  activities/discussions.  Large  group  presentations  were  designed  to  present  state-of- 
the-art  information  on  different  areas  such  as  assessing  local  needs  and  assets,  designing  prevention  programs  to 
meet  desired  results,  and  using  technology  to  maintain  connections  and  locate  resources.  These  presentations  were 
interspersed  with  and  complemented  by  small  group  activities  in  which  coordinators  had  the  opportunity  to  apply 
new  knowledge  or  information  through  discussions  and  case  study  work. 

On-line  Events 

MSCs  were  also  given  the  opportunity  to  participate  in  nine  different  on-line  continuing  education  events 
that  were  held  between  April  2001  and  September  2002.  The  events,  which  each  lasted  between  four  and  five  days, 
were  intended  to:  (1)  deepen  skills  the  MSCs  acquired  during  the  core  training,  and  (2)  foster  the  exchange  of 
information  and  ideas  that  help  transfer  knowledge  into  practice.  USED  urged,  but  did  not  require,  MSCs  to 
participate  in  one  or  more  of  the  events. 

The  events  covered  a wide  range  of  topics  such  as  needs  assessment;  prevention  program  identification, 
selection,  and  implementation;  sustainability;  linking  prevention  to  academic  success;  and  crisis  intervention  (see 
Table  1 for  a list  of  the  titles/topics  of  the  nine  events).  Although  the  content  of  the  events  differed,  they  shared 
many  structural  characteristics.  Each  event  consisted  of  (1)  an  introduction  and  broad  overview  of  the  topics  to  be 
covered,  (2)  a narrative  component  that  either  presented  information  using  examples  from  real  life  situations  or 
adopted  a case  study  approach  where  information  was  imbedded  in  a story  about  a fictional  cast  of  veteran  MSCs, 
(3)  an  event  discussion  area  that  was  facilitated  by  one  or  more  Training  Center  staff  with  expertise  in  the  topic(s) 
covered  by  the  event,  (4)  a summary  of  the  major  discussion  themes  in  the  discussion  area  that  was  updated  at  the 
end  of  each  day  by  the  event  facilitator,  (5)  a set  of  daily  activities,  (6)  links  to  additional  resources,  (7)  a list  of 
references  and  an  annotated  bibliography  of  materials  used  to  generate  the  event  content,  and  (8)  an  event  support 
area  for  MSCs  experiencing  any  technical  difficulties. 

Table  1.  On-Line  Event  Titles/Topics 

On-line  Events  for  Middle  School  Coordinators 

1.  Using  Existing  Data  in  Your  Needs  Assessment 

2.  Identifying  Priorities  and  Strategies  for  Your  Prevention  Initiative 

3.  Promoting  Prevention  Through  School-Community  Partnerships 

4.  Selecting  Research-Based  Prevention  Programs  for  Your  School 

5.  Implementing  Research-Based  Prevention  Programs  in  Schools 

6.  Sustaining  Your  Prevention  Initiative 

7.  Linking  Prevention  to  Academic  Success 

8.  Crisis  Response:  Creating  Safe  Schools 

9.  Middle  School  Coordinators  as  Change  Agents 


Evaluation  Design 

The  US  Department  of  Education  required  the  National  Center  for  Training  and  Technical  Assistance  to 
conduct  a process  and  outcome  evaluation  of  its  services,  including  the  on-line  events.  The  goal  of  the  process 
evaluation  was  to  describe  how  the  program  was  delivered  (e.g.,  participant  characteristics,  satisfaction).  The 
outcome  evaluation  was  designed  to  assess  what  impact(s),  if  any,  the  program  had  (e.g.,  changes  in  knowledge, 
attitudes,  and  skills).  Using  a continuous  quality  improvement  approach,  evaluation  findings  were  fed  back  to  the 
program  staff  and  the  funder  as  they  became  available. 

Evaluation  Instruments  and  Testing  Intervals 

The  primary  source  of  evaluation  data  for  the  on-line  events  was  on-line  surveys  that  were  built  into  each 
event.  The  decision  to  use  on-line  surveys  was  made  for  to  a variety  of  reasons  including:  (1)  the  importance  of 
matching  the  data  collection  method  to  the  program’s  use  of  on-line  learning;  (2)  the  ability  to  incorporate  the 
surveys  directly  into  the  events  so  that  they  would  appear  seamless,  which,  in  turn,  would  help  increase  the  response 
rates,  and  (3)  due  to  the  geographic  distribution  of  MSCs,  on-line  surveying  was  more  cost  effective  than  doing 
multiple  waves  of  mailed  surveys.  It  should  be  noted,  however,  that  there  are  many  occasions  when  on-line  surveys 
are  neither  the  most  appropriate,  nor  the  most  cost  effective  way  of  surveying  one’ s target  population. 
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Due  to  the  timing  of  the  grant,  resources  available,  and  other  factors,  the  data  collection  points  and  the 
research  design  differed  for  the  first  three  events,  the  second  three  events,  and  the  last  three  events.  The  first  three 
on-line  events  were  evaluated  using  an  on-line  posttest  and  a six-month  follow-up  survey.  Beginning  with  the 
fourth  event,  and  continuing  through  the  ninth,  the  evaluators  were  able  to  institute  a pretest  and  posttest  design. 
Three-month  follo\wup  data  were  also  collected  for  the  fourth,  fifth,  and  sixth  events  (see  Table  2). 


Table!.  On-line  Event  Testing  Intervals 


Pretest 

Posttest 

Events  1-3 

Yes 

Yes 

Events  4-6 

Yes 

Yes 

Yes 

Events  7-9 

Yes 

Yes 

Following  participation  in  each  of  the  nine  events,  MSCs  were  asked  to  complete  an  on-line  post-event 
questionnaire.  These  questionnaires  included  items  asking  MSCs  to  rate  different  aspects  of  the  event  (e.g., 
organization  of  the  website,  quality  of  the  materials,  links  to  other  websites,  role  of  the  facilitators),  items  about  the 
usefulness  of  the  information,  items  about  the  likelihood  of  MSCs  recommending  the  event  to  their  peers,  items 
about  overall  satisfaction  with  the  event,  a set  of  ten  knowledge  items,  and  several  open-ended  items  that  allowed 
MSCs  to  report  the  most  and  least  helpful  features  of  the  event  and  to  write  any  additional  comments  or  suggestions. 

Beginning  with  the  fourth  event,  and  continuing  through  the  ninth  event,  MSCs  were  asked  to  complete  an 
on-line  pre -event  questionnaire.  These  questionnaires  included  items  about  the  participants'  background  (e.g.,  years 
of  experience  in  school-based  prevention,  which  of  the  nine  core  trainings  they  had  attended,  which  of  the  previous 
on-line  events  they  had  completed),  a set  of  ten  knowledge  items  about  information  contained  in  the  event,  and  a set 
of  five  behavior  items  about  activities  the  MSCs  had  engaged  in  during  the  three  months  prior  to  the  event. 

Approximately  six-months  after  the  first  three  events,  and  three-months  after  the  fourth  through  sixth 
events,  MSCs  were  asked  to  complete  an  on-line  follow-up  questionnaire.  The  follow-up  questionnaire  asked  MSCs 
to  assess  retrospectively  the  amount  of  information  they  received,  the  usefulness  of  the  information,  the  frequency  of 
using  the  information,  and  the  relevance  of  the  information.  The  knowledge  and  behavior  items  from  the  pretest  and 
posttest  also  appeared  on  the  questionnaire.  Finally,  an  open-ended  item  asked  MSCs  to  provide  one  example  of 
how  they  applied  the  information  from  the  event  to  their  work. 

Response  Rates 

Several  studies  have  found  that  response  rates  for  on-line  surveys  have  been  lower  than  those  obtained 
using  other  methods  such  as  face-to-face  surveys,  telephone  surveys,  and  mailed  surveys  (Couper,  2002).  On 
average,  one  can  expect  to  obtain  response  rates  of  about  80%  to  90%  for  face-to-face  surveys,  85%  for  telephone 
surveys,  and  60%  to  75%  for  mailed  surveys  (Dillman,  1978).  There  are,  however,  a number  of  strategies  that  can 
be  used  to  increase  survey  response  rates.  For  example,  Dillman  (1978)  was  consistently  able  to  achieve  response 
rates  of  nearly  75%  in  mailed  surveys. 

Prior  to  each  of  the  on-line  events,  the  event  was  promoted  by  posting  an  announcement  on  the  MSC 
website,  and  by  mailing  a pamphlet  and  sending  an  e-mail  to  all  of  the  MSCs  who  attended  one  of  the  core  training 
workshops.  MSCs  then  had  an  opportunity  to  register  for  the  event  on  the  MSC  website  through  an  on-line 
registration  system.  For  the  events  that  included  a pretest,  MSCs  were  sent  an  e-mail  invitation  to  complete  the  test 
before  the  event,  and  invitations  were  also  posted  on  the  Introduction  page  and  the  Discussion  Area  of  the  event.  At 
the  end  of  each  event,  MSCs  were  sent  an  e-mail  invitation  to  complete  a posttest.  Also,  invitations  were  posted  on 
the  last  page  of  the  event  and  in  the  Discussion  Area,  and  non -responders  were  sent  an  e-mail  reminder  one  week 
after  the  event.  For  the  three-  and  six-month  follow-up  surveys,  MSCs  were  sent  an  e-mail  invitation,  and  non- 
responders were  sent  up  to  two  more  e-mails  at  one  week  intervals. 

Consistent  with  the  literature,  the  average  pretest  response  rate  was  53%,  the  posttest  survey  response  rate 
was  47%,  the  three-month  follow-up  response  rate  was  54%,  and  the  six-month  follow-up  response  rate  was  65% 
(see  Table  3). 
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Table  3.  On  -line  Event  Response  Rates 


Pretest 

Posttest 

3-Month 
Polio  w- Up 

6-Month 
Polio  w- Up 

Event  1 

84% 

77% 

Event  2 

68% 

61% 

Event  3 

50% 

64% 

Event  4 

36% 

44% 

50% 

Event  5 

43% 

23% 

58% 

Event  6 

67% 

57% 

54% 

Event  7 

66% 

51% 

Event  8 

66% 

53% 

Event  9 

56% 

42% 

Overall  Rates 

53% 

47% 

54% 

65% 

There  are  several  factors  that  may  have  contributed  to  these  comparatively  low,  but  still  respectable  response  rates. 
First,  several  of  the  events  were  conducted  toward  the  end  of  the  academic  year.  This  is  a very  busy  time  for  all 
school  staff,  including  MSCs,  and  the  scheduling  of  these  events  may  have  had  a detrimental  effect  on  the  amount  of 
time  that  MSCs  had  available  to  participate  in  data  collection  activities.  Similarly,  the  follow-up  for  several  of  the 
events  took  place  at  the  beginning  of  the  new  academic  year,  which  can  also  be  a very  busy  time  for  school 
personnel.  Another  factor  that  may  have  influenced  the  follow-up  response  rate  was  that  MSCs  may  have  changed 
jobs  over  the  summer,  their  e-mail  addresses  may  have  changed  at  the  start  of  the  new  school  year,  and  MSCs  from 
the  first  cohort  of  the  grant  might  not  have  been  brought  back  since  their  three-year  position  was  about  to  expire. 

Due  to  limitations  of  the  web-course  software,  it  was  not  possible  to  verify  which  of  the  MSCs  who 
registered  for  the  event  actually  participated  in  the  event.  It  is  possible  that  some  MSCs  may  have  pre -registered  for 
an  event  to  secure  a spot,  but  later  decided  not  to  participate  or  were  unable  to  participate  due  to  scheduling  issues. 
Since  our  denominator  for  each  of  the  response  rates  is  based  on  the  number  of  MSCs  who  registered  for  the  event, 
it  is  likely  that  the  response  rates  in  Table  3 are  under-estimates  of  the  MSCs  who  responded  and  who  actually  took 
part  in  the  event. 

Evaluation  Questions 

A fundamental  research  principle  is  that  what  you  learn  depends  on  what  you  ask  and  how  you  ask  it. 
Essentially,  your  purpose  should  guide  your  decisions  about  what  data  to  collect.  Before  collecting  any  data,  you 
should  ask  yourself  three  questions:  (1)  what  do  I want  to  know,  (2)  why  do  I want  to  know  it,  and  (3)  what  would  I 
do  if  I did  know  it?  While  it  is  fairly  easy  to  conduct  an  on-line  survey,  especially  with  all  the  new  on-line  software 
packages  available,  it  is  difficult  to  create  a good  survey.  There  is  a large  body  of  literature  on  survey  and 
questionnaire  design,  and  just  because  a survey  is  conducted  on-line  does  not  mean  that  these  rules  no  longer  apply. 
Before  creating  a survey,  a social  scientist  or  someone  familiar  with  the  literature  should  be  consulted.  Something 
as  ''simple''  as  the  way  a question  is  worded  can  have  a profound  impact  on  the  data  and  the  methods  available  to 
analyze  the  data. 

As  mentioned  above,  process  evaluation  data  are  used  to  describe  how  the  program  was  delivered  (e.g., 
participant  characteristics,  participant  satisfaction),  and  outcome  evaluation  data  are  used  to  assess  what  impact(s),  if 
any,  the  program  had  (e.g.,  changes  in  knowledge,  attitudes,  and  skills).  In  this  study,  process  evaluation  questions 
included:  (1)  who  is  being  served  (e.g.,  the  number  of  people,  demographic  information,  technical  expertise),  (2) 
who  is  not  being  served?  (3)  are  participants  satisfied  with  the  events,  and  (4)  are  there  ways  that  the  process  can  be 
modified  to  make  it  better?  Outcome  evaluation  questions  included:  (1)  what  are  the  short-term  impacts,  (2)  what 
are  the  long-term  impacts,  and  (3)  is  the  pedagogy  making  the  most  effective  use  of  resources? 

Who  is  Being  Served? 

One  of  the  most  basic  issues  an  evaluation  can  address  is  to  describe  the  people  who  are  being  served.  This 
can  range  from  very  simple,  but  very  important,  information  on  the  number  of  people  being  served  (for  example, 
56%  of  the  924  MSCs  who  participated  in  the  face-to-face  core  trainings  participated  in  one  or  more  of  the  on-line 
events),  to  more  descriptive  information  about  the  participants’  gender,  age,  race,  and  other  characteristics.  In 
addition  to  demographic  information,  the  pretest  questionnaires  asked  MSCs  to  report  on  background  characteristics 
such  as,  number  of  years  experience  in  school-based  prevention  (including  any  experience  from  before  they  were 
hired  as  an  MSC),  the  date  when  they  began  their  position  as  an  MSC,  the  core  training  workshop  in  which  they  had 
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participated,  the  Internet  connection  speed  for  the  computer  they  most  often  used  to  access  the  event,  whether  or  not 
they  had  participated  in  an  Introductory  event  that  was  provided  to  all  MSCs  to  teach  them  how  successfully 
navigate  the  on-line  events  and  assessed  whether  or  not  their  software  was  capable  of  running  all  of  the  features  of 
the  event,  and  in  which  other  events,  if  any,  they  had  also  participated. 

Demographic  and  background  information  is  valuable  for  both  the  funding  agency  and  for  the  service 
provider.  For  example,  demographic  information  on  race/ethnicity  can  help  a service  provider  determine  whether  or 
not  the  event  materials  are  culturally  appropriate  for  all  of  the  participants,  and  information  on  participants'  previous 
experience  using  the  Internet  can  help  forecast  potential  problem  aieas  that  will  require  technical  assistance.  This 
type  of  information  is  also  useful  when  dealing  with  a sample  of  people  from  a larger  population  (as  was  the  case 
with  the  MSC  on-line  events).  By  collecting  demographic  and  background  information,  it  is  possible  determine  how 
well  the  people  you  are  serve  represent  the  larger  group.  Demographic  and  background  information  can  also  be 
used  to  compare  the  outcomes  (such  as  change  in  knowledge,  attitudes,  and  skills)  for  different  subgroups  of 
participants.  For  example  you  could  ask  if  the  on-line  events  produced  similar  outcomes  for  MSCs  with  more 
versus  less  experience  in  school-based  prevention. 

Who  is  Not  Being  Served? 

Although  the  opportunity  to  collect  data  about  people  who  do  not  receive  services  does  not  often  present 
itself,  this  type  of  information  can  be  used  to  identify  any  barriers  to  participation  (e.g.,  lack  of  time,  unfamiliar  with 
the  use  of  web  technology).  In  this  study,  since  all  of  the  MSCs  were  required  to  participate  in  one  of  the  core 
training  workshops,  it  was  possible  to  identify  MSCs  who  did  not  participate  in  any  of  the  first  three  events  that 
were  offered.  Following  the  delivery  of  the  first  three  on-line  events,  269  MSCs  who  did  not  participate  in  one  of 
the  events  were  sent  an  e-mail  invitation  to  participate  in  an  on-line  survey  designed  to  assess  the  main  reason  that 
they  chose  not  to  participate  and  to  assess  the  likelihood  of  their  participating  in  future  events.  Of  the  269  MSCs 
who  received  an  invitation,  175  MSCs  (65%)  completed  the  questionnaire. 

The  first  item  in  the  questionnaire  asked  about  the  main  reason  MSCs  chose  not  to  participate.  It  included 
six  response  options:  (1)  I cannot  access  the  Internet  (no  computer,  software,  provider,  knowledge),  (2)  I did  not 
have  enough  time  to  participate,  (3)  I registered  for  an  event  that  was  cancelled,  (4)  I do  not  like  the  idea  of  taking 
on-line  courses,  (5)  I was  not  interested  in  the  topics,  and  (6)  I did  not  know  about  the  events.  MSCs  were  also 
given  the  option  of  writing  in  another  reason  that  did  not  appear  on  the  list.  The  most  frequent  responses  given  were 
that  they  either  did  not  have  enough  time  to  participate  (50%),  or  that  they  had  registered  for  a later  event  that  was 
cancelled  (24%).  Additional  comments  fro  m the  open-ended  option  seemed  to  corroborate  these  two  themes  (e.g., 
scheduling  conflicts  with  the  course  of  interest,  failed  to  make  time  to  participate,  anticipated  registering  for  an 
event  that  was  later  cancelled).  Very  few  MSCs  selected  the  more  “negative”  options  such  as,  “I  do  not  like  the  idea 
of  taking  courses  on-line”  (8%),  or  “I  was  not  interested  in  the  topics”  (5%).  This  was  important  information, 
because  it  suggested  that  MSCs  who  decided  not  to  participate  in  the  first  three  events  did  not  do  so  because  of 
dissatisfaction  with  the  topics  or  an  aversion  to  using  the  Internet  as  a medium  for  information  dissemination 
(Formica  and  Harding,  2001a). 

Another  item  asked  MSCs  to  indicate  the  likelihood  of  their  participating  in  future  on-line  events  using  a 
five-point  scale:  Not  At  All  Likely,  Not  Very  Likely,  Somewhat  Likely,  Fairly  Likely,  and  Very  Likely.  Eighty- 
nine  percent  (89%)  of  respondents  indicated  that  they  were  either  “Somewhat  Likely”  (28%),  Fairly  Likely  (36%), 
or  “Very  Likely”  (25%)  to  participate  in  future  on-line  events.  The  finding  that  only  12%  of  respondents  indicated 
that  they  were  not  likely  (“Not  At  All  Likely”  or  “Not  Very  Likely”)  to  participate  in  future  on-line  events  was 
consistent  with  the  findings  from  the  previous  item.  The  reason  for  non-participation  seemed  to  center  around  lack 
of  time  as  opposed  to  a less  desirable  alternative  such  as  lack  of  interest  or  skill.  This  information  was  used  as  a 
basis  for  a decision  to  shorten  the  length  of  the  on-line  events  from  five  days  to  four  days  in  an  attempt  to  be  more 
accommodating  to  MSCs  with  busy  schedules  and  to  ameliorate  a perceived  barrier  to  participation. 

Are  Participants  Satisfied  with  the  Services? 

Process  data  about  participant  satisfaction  with  different  comp  onents  of  an  on-line  event  (e.g.,  quality  of 
the  information,  satisfaction  with  the  facilitators)  can  point  to  ways  the  event  design  might  be  improved.  Collecting 
data  on  overall  satisfaction  with  the  event  can  help  predict  whether  people  are  likely  to  participate  in  future  events. 

For  the  MSC  project,  posttest  items  asked  participants  to  rate  their  satisfaction  with  several  different 
components  of  the  event  (organization  and  layout  of  the  event  website,  quality  of  materials,  role  of  the  facilitators, 
links  provided  to  other  websites,  download  speed  of  the  webpages),  and  their  overall  satisfaction  with  the  event  on  a 
five  point  scale:  Very  Dissatisfied,  Somewhat  Dissatisfied,  Neutral,  Somewhat  Satisfied,  and  Very  Satisfied.  The 
percent  of  MSCs  who  indicated  that  they  were  either  “Somewhat  Satisfied”  or  “Very  Satisfied”  with  the  on-line 


152 


events  ranged  from  79%  to  97%  (see  Table  4).  Overall,  across  all  nine  events,  92%  of  posttest  respondents 
indicated  that  they  were  either  “Very  Satisfied”  (62%),  or  “Somewhat  Satisfied”  (30%).  Only  8%  of  respondents 
indicated  that  they  were  “Neutral”  (4%),  “Somewhat  Dissatisfied”  (3%),  or  “Very  Dissatisfied”  (1%).  These  high 
ratings  indicated  there  was  not  a need  to  modify  any  event.  Satisfaction  with  different  components  of  the  ondine 
events  also  received  high  ratings,  indicating  that  it  was  not  necessary  to  modify  their  components. 


Table  4.  On-Line  Event  Satisfaction  Rates 


Very 

Dissatisfied 

Somewhat 

Dissatisfied 

Neutral 

Somewhat 

Satisfied 

Very 

Satisfied 

Event  1 

6% 

2 

6% 

2 

6% 

2 

39% 

14 

44% 

16 

Event  2 

4% 

3 

5% 

4 

12% 

10 

41% 

33 

38% 

31 

Event  3 

2% 

1 

6% 

3 

4% 

2 

39% 

21 

50% 

27 

Event  4 

4% 

4 

1% 

1 

3% 

3 

23% 

21 

69% 

64 

Event  5 

2% 

1 

3% 

2 

5% 

3 

28% 

16 

62% 

36 

Event  6 

1% 

1 

1% 

1 

1% 

1 

28% 

20 

68% 

48 

Event  7 

1% 

1 

1% 

1 

2% 

2 

31% 

29 

65% 

62 

Event  8 

3% 

2 

1% 

1 

0% 

0 

17% 

13 

79% 

61 

Event  9 

0% 

0 

3% 

2 

0% 

0 

30% 

21 

67% 

46 

Overall  Rates 

1% 

15 

3% 

17 

4% 

23 

30% 

188 

62% 

391 

Indirect  satisfaction  data  can  also  come  from  other  sources  such  as  participation  records  (e.g.,  how  many 
people  participated,  do  people  participate  in  more  than  one  event  if  there  are  multiple  events),  and  questions  about 
whether  or  not  people  would  recommend  the  service  to  their  peers.  For  example,  over  half  (56%)  of  the  924  MSCs 
who  participated  in  the  face-to-face  core  trainings  participate  in  one  or  more  of  the  on-line  events,  and  75%  of  the 
MSCs  who  participated  in  an  on-line  event  participated  in  more  than  one  event.  MSCs  were  asked  whether  or  not 
they  would  recommend  the  event  to  other  MSCs  on  a five  point  scale:  Strongly  Recommend  They  Not  Participate, 
Recommend  They  Not  Participate,  Neutral,  Recommend  They  Participate,  Strongly  Recommend  They  Participate. 
Across  all  nine  events,  95%  of  posttest  respondents  indicated  that  they  would  either  Strongly  Recommend  (53%)  or 
Recommend  (42%)  the  event  to  other  MSCs,  and  an  additional  5%  indicated  that  they  were  Neutral.  Amazingly, 
only  one  MSC  indicated  that  they  would  “Not  Recommend”  the  event,  and  none  of  the  MSCs  in  any  of  the  events 
indicated  that  they  would  “Strongly  Not  Recommend”  the  event  to  other  MSCs.  In  addition  to  serving  as  a good 
litmus  test  of  the  quality  of  the  services  that  are  being  provided,  this  information  is  also  useful  for  marketing 
services  to  future  participants  and  to  funders. 

Are  There  Ways  the  Process  Can  Be  Modified  to  Make  It  Better? 

Open-ended  write-in  questions  can  be  used  to  assess  whether  participants  have  any  specific  suggestions  for 
modifying  the  delivery  of  on-line  services.  In  this  study,  MSCs  were  asked  three  open-ended  questions  at  the  end  of 
the  posttest  questionnaire:  (1)  what  were  the  most  helpful  features  of  the  event,  (2)  what  were  the  least  helpful 
features  of  the  event,  and  (3)  please  write  any  additional  comments  or  suggestions.  These  data  were  used  to  identify 
the  strengths  and  weaknesses  of  the  events  that  were  not  apparent  to  the  event  designers.  For  example,  as  a result  of 
information  generated  from  responses  to  these  items  given  in  the  first  three  events,  the  entire  design  of  the  course 
website  was  changed  to  increase  the  size  of  the  primary  event  window,  to  reduce  the  size  of  toolbars,  and  to  relocate 
the  navigation  bars.  While  open-ended  questions  can  be  a very  rich  source  of  data,  they  should  be  used  sparingly, 
particularly  with  large  samples,  because  it  is  very  costly  and  time  consuming  to  analyze  and  present  the  qualitative 
data  they  provide. 

What  Are  the  Short-Term  Impacts? 

Probably  the  most  important  evaluation  issue  is  whether  an  on-line  service  has  any  positive  impact.  Did 
the  people  who  participated  in  the  MSC  events  gain  anything  as  a result  of  their  participation?  In  the  MSC  project, 
one  method  for  measuring  short-term  impact  (immediate  effects  of  the  events)  was  to  use  a set  of  approximately  10 
knowledge  questions  per  event  that  represented  the  content  of  the  event.  These  same  questions  were  asked  on  the 
pretest  and  on  the  posttest,  and  analyzed  to  determine  whether  there  were  any  statistically  significant  differences 
between  these  two  testing  points. 

For  the  six  events  on  which  both  pretest  and  posttest  data  were  collected,  paired  t-tests  were  used  to  assess 
differences  between  the  knowledge  items  at  the  time  of  the  pretest  and  the  time  of  the  posttest.  Results  indicated 
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that  MSCs  made  gains  on  between  eight  and  ten  items  out  of  the  10  knowledge  questions  in  any  given  event.  In 
addition,  many  of  these  differences  were  statistically  significant  (p<.05),  indicating  that  these  differences  were  most 
likely  not  due  to  chance  alone.  Given  the  consistency  of  these  findings,  it  is  reasonable  to  assume  that  even  mo  re  of 
these  differences  might  have  achieved  statistical  significance  with  a larger  sample  of  respondents.  Overall,  these 
data  supported  the  conclusion  that  MSCs  were  making  short-term  gains  (increased  knowledge)  as  a result  of 
participating  in  the  events  (Formica  and  Harding,  2001b;  Formica  and  Harding  2002a). 

What  Are  the  Long-Term  Impacts? 

Information  about  long-term  impact  is  often  very  difficult  to  collect  because  participants  may  have  moved, 
switched  positions,  or  changed  contact  information.  However,  when  it  is  possible  to  collect  it,  this  is  very  powerful 
information  because  it  can  be  used  to  assess  whether  or  not  participants  have  changed  their  behavior  (which  often 
cannot  be  determined  in  the  short  time  period  between  the  pretest  and  posttest),  and  whether  any  knowledge  gained 
as  a result  of  the  services  provided  has  been  retained. 

In  the  MSC  project,  long-term  impact  was  assessed  by  asking  MSCs  to  report  at  the  time  of  the  pretest  and 
again  at  follow-up  on  a set  of  behaviors  that  they  would  be  expected  to  have  engaged  in  as  a result  of  the  lessons 
learned  in  the  events  (e.g.,  convening  a group  of  school  and  community  stakeholders  to  help  select  an  appropriate 
prevention  program).  Data  from  the  three  events  where  there  was  both  a pretest  and  a follow-up  questionnaire 
indicated  that  MSCs  were  more  likely  to  indicate  that  they  had  taken  part  in  the  desired  behavior  in  the  three  months 
following  completion  of  the  event  than  they  were  in  the  three  months  before  the  event.  Specifically,  MSCs  showed 
change  in  the  preferred  direction  on  three  of  five  items  in  the  fourth  event  (all  3 were  significant),  on  all  five  items  in 
the  fifth  event  (all  5 were  significant),  and  on  all  five  items  in  the  sixth  event  (3  of  which  were  significant).  These 
results  provided  strong  evidence  that  MSCs  were  likely  to  engage  in  desired  behaviors  following  their  participation 
in  these  events  (Formica  and  Harding,  2002a). 

To  assess  long-term  impact  we  also:  (1)  repeated  knowledge  items  on  the  follow-up  questionnaire  and 
compared  them  to  the  posttest  in  order  to  determine  whether  or  not  there  were  any  significant  declines  in  knowledge 
or  whether  it  was  retained;  and  (2)  used  open-ended  items  to  obtain  anecdotal  information  about  whether  and  how 
MSCs  applied  information  gained  through  the  on-line  event.  In  general,  results  of  analyses  conducted  on  the 
knowledge  items  from  posttest  to  follow-up  indicated  that  MSCs  did  not  show  any  significant  declines  in  knowledge 
at  either  three  or  six  months  following  participation  in  the  events  (Formica  and  Harding,  2002a;  Formica  and 
Harding  (2002b).  In  addition,  virtually  all  of  the  MSCs  who  responded  to  the  follow-up  surveys  were  able  to 
provide  at  least  one  example  of  how  they  applied  information  from  the  event  to  their  work  as  an  MSC  (Formica  and 
Harding,  2002a).  For  example,  in  response  to  this  item  on  the  follow-up  survey  for  the  fourth  event  on  selecting 
research-based  programs,  one  MSC  wrote,  “I  put  together  an  Advisory  Committee  consisting  of  school,  community, 
and  youth  resources  to  review  prevention  efforts,  identify  strengths  and  weaknesses,  review  prevention  strategies, 
and  select  programs  that  met  our  needs  and  abilities.  We  are  now  implementing  a program.”  As  with  the 
satisfaction  data,  these  success  stories  can  again  be  used  to  market  an  on-line  service  to  a funder  and  to  future 
participants.  And  anecdotal  data  such  as  this  can  sometimes  have  a more  profound  effect  on  the  reader  of  an 
evaluation  report  than  pages  full  of  data. 

Is  the  Pedagogy  Making  the  Most  Efficient  Use  of  Resources? 

Outcome  evaluation  data  can  also  be  used  to  help  determine  the  answers  to  such  questions  as  whether  or 
not  the  method  of  information  dissemination  is  making  the  most  efficient  use  of  time  and  resources.  For  the  first 
three  on-line  events,  based  on  research  indicating  that  large  numbers  of  participants  in  distance  learning  events 
might  be  detrimental  to  the  quality  of  services,  the  events  adopted  a two-tiered  model  of  participation  (Giguere, 
Formica,  & Harding,  2002;  Ko  & Rossen,  2001;  Palloff  & Pratt,  1999).  In  this  model,  MSCs  were  able  to  register  as 
either  an  active  participant  or  as  an  auditor.  Active  participants  were  required  to  log  on  to  the  event  at  least  once  a 
day,  review  event  materials,  participate  in  event  activities,  and  contribute  to  the  facilitated  discussion  area.  Auditors 
were  not  required  to  log  on  at  lease  once  a day,  were  not  required  to  participate  in  event  activities,  and  were  able  to 
view  the  facilitated  discussion  but  could  not  contribute  to  it.  Only  50  MSCs  were  able  to  register  as  active 
participants,  the  remaining  registrants  were  placed  into  the  auditor  category.  The  rationale  for  this  model  was  to  be 
able  to  be  able  to  provide  the  event  to  all  MSCs,  but  to  not  overwhelm  the  event  facilitator  in  the  discussion  area  and 
to  allow  the  opportunity  for  more  meaningful  interaction  with  the  facilitator  than  might  be  possible  with  large 
numbers  of  participants. 

Surprisingly,  results  from  analyses  comparing  active  participants  versus  auditors  found  that  the  differences 
between  these  two  groups  were  both  small  and  statistically  not  significant  for  the  amount  of  information  they  felt 
they  acquired,  their  ratings  of  the  usefulness  of  the  information  they  received,  self-reported  time  spent  participating 
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in  the  events,  the  true/false  items  tailored  to  each  event,  overall  satisfaction  with  the  event,  or  willingness  to 
recommend  the  event  to  other  MSCs  (Formica  & Harding,  2001b).  Given  the  lack  of  differences,  it  was  determined 
that  the  events  should  be  open  to  all  MSCs  at  the  same  level  of  participation.  In  response  to  concerns  from  the  event 
facilitators  that  they  would  not  be  able  to  adequately  respond  to  all  of  the  messages  in  the  Discussion  Area  an 
additional  staff  member  was  assigned  to  help  deal  with  the  increase  in  messages.  While  the  direct  effect  of  this 
model  change  was  not  systematically  evaluated,  participant  satisfaction  ratings  did  not  show  decline  following  this 
change  (see  Table  4),  and  the  short-term  and  long-term  impact  measures  did  not  seem  to  be  detrimentally  affected. 

In  short,  this  change  allowed  the  Center  to  serve  more  MSCs  in  a more  intensive  manner  without  negatively 
affecting  the  quality  of  the  services  provided.  This  approach  was  also  more  time  and  resource  effective  than  having 
to  repeat  an  event  multiple  times.  Another  indirect  effect  of  this  change  was  that  the  time  and  resources  saved  were 
channeled  into  the  creation  of  additional  events. 

Discussion 

While  most  often  used  to  conduct  marketing  research  and  website  satisfaction,  the  authors  found  that  the 
use  of  on-line  surveys  can  be  a useful  way  to  conduct  a process  and  outcome  evaluation  of  large-scale  national 
training  efforts  such  as  the  MSC  project.  Results  indicate  that  respectable  response  rates  can  be  achieved  using  this 
method  when  adopting  proven  strategies  traditionally  used  to  increase  response  rates  to  mailed  surveys.  In  addition, 
when  dealing  with  a geographically  distributed  population,  on-line  surveying  can  be  a more  cost  effective  method 
than  conducting  multiple  waves  of  mailed  surveys.  On-line  surveys  also  hold  other  less  obvious  advantages  such  as 
reduced  cost  of  data  entry  and  increased  accuracy  due  to  the  elimination  of  data  entry  and  data  entry  error. 

This  paper  also  emphasizes  the  importance  of  defining  process  and  outcome  evaluation  questions  at  the 
beginning  of  a project  to  enable  the  use  of  a continuous  quality  improvement  approach  where  evaluation  findings  on 
how  the  program  was  delivered  and  what  impact(s),  if  any,  the  program  had  can  be  fed  back  to  program  staff  and  the 
funder.  By  applying  this  framework  up  front  and  using  three  basic  guiding  questions:  (1)  what  do  I want  to  know, 
(2)  why  do  I want  to  know  it,  and  (3)  what  would  I do  if  I did  know  it,  the  evaluation  can  provide  meaningful  data  to 
help  inform  the  delivery  of  services  and  the  ability  to  adequately  report  findings  at  the  end  of  the  project. 
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Abstract 

This  survey  was  carried  out  on  behalf  of  the  Association  for  Educational  Communications  and  Technology 
(AECT).  The  2001  survey  assessed  the  needs  of  AECT’ s members  and  identified  the  Association’s  weaknesses.  A 
total  of 553  usable  survey  responses  were  received.  According  to  the  quantitative  data  gathered,  most  respondents 
were  on  the  whole  satisfied  by  the  services  provided  by  AECT  with  regard  to  conferences,  publications,  and  online 
services.  Members  appeared  to  be  more  aware  of  certain  online  services  as  compared  with  others.  However,  the 
vast  majority  of  those  who  had  used  any  of  the  online  facilities  were  generally  satisfied  with  their  experience. 
Meanwhile,  the  qualitative  comments  expressed  by  the  respondents  provide  a more  rich  and  complex  picture, 
describing  specific  situations,  problems,  and/or  issues  that  provide  insight  into  the  quality  of  AECT  services. 

Introduction  to  AECT 

The  Association  for  Educational  Communications  and  Technology  (AECT)  is  a professional  association  of 
thousands  of  educators  and  others  whose  activities  are  directed  towards  improving  instruction  through  the  use  of 
technology.  Established  in  1923,  AECT  has  become  a major  organization  for  those  who  are  actively  involved  in 
designing  instruction  and  utilizing  a systematic  approach  to  learning.  Providing  an  international  forum  for  the 
exchange  and  dissemination  of  ideas  for  its  members  and  for  larger  audiences,  AECT  is  dedicated  to  the 
improvement  of  instruction.  It  is  the  world's  largest  publisher  of  information  concerning  instructional  technology 
(AECT  History,  2001).  The  Association’s  mission  is  to  provide  leadership  in  educational  communications  and 
technology  by  linking  professionals  who  hold  a common  interest  in  the  use  of  educational  technology  and  its 
application  to  the  learning  process  (Mission  and  Goals,  2001). 

AECT  Members 

The  Association’s  3000+  members  and  subscribers  work  in  academia,  the  armed  forces,  industry, 
museums,  libraries,  and  hospitals.  AECT  members  carry  out  a wide  range  of  responsibilities  in  the  study,  planning, 
application,  and  production  of  mediated  communications  for  instruction  (AECT  History,  2001).  Its  members  enjoy 
a wide  range  of  benefits,  including:  free  subscription  to  the  TechTrends  journal;  discounts  for  subscription  to 
Educational  Technology  Research  & Development  (ETR&D),  Distance  Quarterly  Review,  and  other  AECT 
publications;  access  to  numerous  online  services  (including  e-newsletter,  membership  directory,  online  publications, 
and  listservs);  discounts  to  the  annual  conference  and  summer  leadership  institute;  and  lower  rates  on  automobile 
insurance  (Membership  Benefits,  2001). 

AECT  Conferences 

Conferences  are  an  important  function  of  AECT.  It  initially  held  national  conferences  in  conjunction  with 
the  National  Education  Association  (NEA).  Then,  in  1952,  AECT  held  a separate  conference  from  the  NEA  for  the 
first  time  and  by  1971  was  no  longer  an  official  department  of  the  NEA.  Between  1982-1984,  a major  change  took 
place  in  the  configuration  of  the  national  conference.  This  is  when  AECT  sponsored  an  international  exposition  of 
communication  materials  and  devices  along  with  the  National  Audio  Visual  Association  (NAVA).  This  exposition, 
called  COMMTEX  International,  was  later  known  as  INEOCOMM  International  due  to  co-sponsorship  by  the 
International  Communication  Industries  Association  or  ICIA.  In  1994,  AECT  broke  away  from  the  ICIA  and  held  its 
own  trade  show,  known  as  InCITE  until  1999  (Erick,  Duvenci,  Kim,  Richter,  & Yang,  2001).  The  timing  of  the 
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Conference  was  then  changed  in  2000  from  February  to  November  in  order  to  coincide  with  the  National  School 
Board  Association  (NSBA)  trade  show. 


Introduction  to  the  2001  Survey 

The  following  research  questions  were  identified  following  a literature  review  of  past  AECT  member 
surveys  and  interviews  with  the  AECT  leadership  regarding  the  vital  aspects  of  the  members  and  their  needs: 

1.  What  are  the  typical  demographics  of  AECT  members? 

2.  What  AECT  services  are  members  aware  of  and  using? 

3.  What  services  provided  by  AECT  (particularly  with  regard  to  publications,  conferences,  and  the  AECT 
Web  site)  are  satisfactory  to  members? 

4.  What  services  provided  by  AECT  (particularly  with  regard  to  publications,  conferences,  and  the  AECT 
Web  site)  are  unsatisfactory  to  members? 

Methodology 

We  decided  to  survey  the  entire  population  of  past  and  current  Association  members  as  listed  in  the  AECT 
database.  Eollowing  a literature  review  of  past  AECT  member  surveys  and  interviews  with  the  Association’s 
Executive  Director,  the  survey  questionnaire  instrument  was  prepared.  In  order  to  carry  out  data  collection  in  the 
most  efficient  and  economical  manner,  a Web-based  survey  instrument  was  used.  Additionally,  a small  percentage 
of  respondents  filled  out  paper-based  submissions  during  the  2001  Conference  in  Atlanta,  Georgia. 

Data  Analysis  and  Reporting 

When  we  began  to  work  with  the  data,  there  were  a total  of  553  usable  responses.  We  analyzed  the 
quantitative  data  (i.e.  those  that  were  generated  by  radio  buttons  and  check  boxes)  using  the  SPSS  statistical  software 
package.  Simultaneously,  we  did  a content  analysis  of  the  qualitative  data  generated  by  open-ended  questions  using 
two  independent  coders. 


Findings 

Survey  Respondents 

Roughly  94  percent  of  the  553  usable  responses  were  made  up  of  online  submissions;  only  six  percent  of 
the  respondents  submitted  a paper  version  at  the  2001  Atlanta  conference.  Nearly  94  percent  of  the  respondents  were 
active  and  current  AECT  members,  only  around  two  percent  were  non-members,  and  the  rest  were  not  sure  of  their 
membership  status.  Meanwhile,  the  biggest  chunk  of  respondents  (35  percent)  had  been  AECT  members  for  around 
one  year  at  the  time  of  the  survey  and  around  92  percent  of  the  respondents  resided  within  the  United  States.  Higher 
education  was  the  biggest  field  among  the  survey  respondents  with  around  70  percent  being  associated  with  higher 
education,  either  as  faculty  or  as  students.  Other  respondents  included  those  from  K- 12,  from  private  business/non- 
profit sectors,  and  those  who  were  retirees. 

Services  Most  Beneficial  and  in  Most  Need  of  Improvement 

As  can  be  seen  from  Eigure  1,  the  annual  conference,  access  to  publications  and  networking  were  perceived 
to  be  the  most  beneficial  services  provided  by  AECT.  Online  services  and  professional  development  were  also  seen 
as  significant  benefits  of  AECT  membership.  Meanwhile,  the  annual  conference  and  online  services  were  also  voted 
by  the  respondents  to  be  the  aspects  most  in  need  of  improvement  among  the  services  provided  by  AECT  (see 
Eigure  2).  Other  services  significantly  in  need  of  improvement  were  communication  between  members  and 
administration,  professional  development,  access  to  publications,  and  organizational  restructuring. 
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FIGURE  1:  Services  Most  Beneficial  to  Members  («=553) 
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FIGURE  2:  Services  Most  in  Need  of  Improvement  («=553) 
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Qualitative  comments  by  respondents  also  mentioned  a need  for  improvement  with  regard  to  leadership, 
focus,  networking,  resources,  employment,  administration,  marketing,  lobbying,  etc.  Many  of  these  comments 
expressed  severe  disappointment  with  the  Atlanta  conference,  particularly  the  scheduling  of  presentations  and  the 
high  expense.  Timing  the  survey  right  after  the  conference  appears  to  have  added  to  these  conclusions.  Several 
comments  also  stressed  a need  for  increased  focus  on  the  part  of  the  Association.  Other  comments  provided 
feedback  regarding  AECT’s  on-line  services,  publications,  and  communications  between  AECT  leadership  and 
members. 

Opinions  Regarding  AECT  Publications 

AECT  seemed  to  be  performing  adequately  with  respect  to  its  flagship  publications,  since  nearly  7 1 percent 
of  respondents  (m=469)  indicated  being  either  satisfied  or  very  satisfied  with  TechTrends  and  around  8 1 percent  of 
respondents  («=343)  indicated  likewise  for  ETR&D. 

Opinions  Regarding  AECT  Conferences 

Out  of  the  three  recent  AECT  annual  conferences,  most  respondents  had  attended  the  Atlanta  conference  in 
November  2001  (53%  of  «=553),  followed  by  Long  Beach  in  2000  (28%  of  n=553),  and  then  Denver  (26%  of 
n=553).  Figure  3.1  shows  types  of  activities  respondents  found  helpful  for  meeting  people  at  conferences. 
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Meanwhile,  through  qualitative  comments,  a number  of  respondents  stressed  how  much  they  liked  having  an  area 
with  tables  and  chairs  in  the  lobby  area  of  the  hotel  for  informal  meetings  between  sessions. 

FIGURE  3.1:  Conference  Activities  Helpful  for  Meeting  People  («=356) 


AECT  changed  the  timing  of  its  Annual  Conference  in  2000  from  Eebruary  to  November  in  order  to 
coincide  with  the  NSBA  trade  show.  The  majority  of  the  respondents  (75%  of  «=478)  agreed  with  this  change. 
Also,  nearly  59  percent  of  the  respondents  felt  that  the  trade  show  was  important  or  very  important  for  the 
conference  (see  Eigure  3.2). 

EIGURE  3.2:  Importance  of  Trade  Show  for  Conference  («=509) 
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Opinions  Regarding  AECT’s  Online  Services 

Ninety-five  percent  of  the  respondents  («=550)  said  they  have  visited  the  AECT  web  site  at: 
http://www.aect.org.  Sixty-eight  percent  («=541)  responded  that  they  have  tried  to  log  on  to  the  site  as  a member,  of 
which  eighty-seven  percent  («=349)  managed  to  log  on  successfully. 

Eifty-six  percent  of  the  respondents  («=548)  were  aware  of  the  web  site’s  online  membership  directory. 
Those  who  had  used  the  directory  had  done  so  to  change/update  their  own  information,  or  to  search  for  another 
member’s  information  (see  Eigure  4). 
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FIGURE  4:  Usage  of  Online  Directory  (n=305) 


Sixty-seven  percent  of  the  respondents  (n=507)  were  not  aware  of  the  availability  of  ETR&D  in  electronic 
format  on  the  web  site.  Ninety  percent  of  these  said  they  would  use  it  had  they  been  aware.  Of  those  who  read 
ETR&D  online,,  around  74  percent  were  satis  fied  (or  very  satisfied)  with  it  (see  Figure  5). 
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Similarly,  the  majority  of  respondents  (around  56  percent  of  «=516)  were  not  aware  of  the  availability  of 
books  in  electronic  format  on  the  web  site,  but  said  they  would  have  read  them  online  if  they  knew  of  their 
existence.  Most  (roughly  80  percent)  of  those  who  had  read  books  online  expressed  satisfaction  with  them  (see 
Figure  6).  The  same  trend  was  also  seen  with  regard  to  purchasing  books  electronically  (see  Figure  7). 
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FIGURE  6:  Satisfaction  with  Books  Online  (n=  1 19) 
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FIGURE  7:  Satisfaction  with  Purchasing  Books  Online  (n=76) 
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Meanwhile,  the  majority  of  respondents  (around  58  percent  of  «=496)  was  indeed  aware  of  the  availability 
of  online  conference  registration  facilities  on  the  Web  site  and  used  these  facilities.  Twenty-eight  percent  were 
aware,  but  did  not  use  the  online  registration.  Most  (roughly  78  percent)  of  those  who  had  registered  online 
expressed  satisfaction  with  the  ejqterience  (see  Figure  8). 
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FIGURE  8:  Satisfaction  with  Registering  for  Conference  Online  (n=285) 
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On  the  other  hand,  while  the  majority  of  respondents  (around  60  percent  of  «=498)  were  aware  of  the 
availability  of  online  registration  for  the  summer  leadership  meeting,  more  than  three-fourths  of  the  498  respondents 
did  not  use  this  facility.  Once  again,  most  (roughly  70  percent)  of  those  who  had  registered  online  expressed 
satisfaction  with  the  experience  (see  Figure  9). 

FIGURE  9:  Satisfaction  with  Registering  for  Summer  Leadership  Meeting  Online  («=61) 
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Regarding  submitting  new  membership  applications  via  the  Web  site,  most  respondents  (69  percent  of 
«=490)  were  aware  of  this  option  and  were  almost  equally  divided  into  those  that  used  this  option  and  those  that  did 
not  use  it.  Of  those  that  did  submit  new  membership  applications  online,  most  (78  percent)  expressed  satisfaction 
(see  Figure  10).  The  exact  same  trend  was  observed  with  respect  to  respondents’  use  of  the  Web  site’s  facilities  to 
renew  memberships  online  (see  Figures  1 1 and  12). 
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FIGURE  10:  Satisfaction  with  Submitting  New  Membership  Application  Online  («=182) 
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FIGURE  11:  Online  Feature  (Renewing  Membership  Electronically)  (n=491) 


FIGURE  12:  Satisfaction  with  Renewing  Membership  Online  (n=177) 
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When  it  came  to  submitting  conference  presentation  proposals  through  the  Web  site,  the  majority  of 
respondents  (47  percent  of  «=491)  were  aware  oiand  used  this  facility.  Of  these,  most  (nearly  84  percent)  expressed 
satisfaction  with  the  experience  (see  Figure  13). 

FIGURE  13:  Satisfaction  with  Submitting  Conference  Presentation  Proposals  Online  («=230) 
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With  respect  to  reading  job  postings  electronically,  the  findings  showed  significant  proportions  of  the 
respondents  (35  percent  each)  divided  into  the  “not  aware,  but  would  use”  and  “aware,  and  use”  categories  (see 
Figure  14).  But  once  again,  most  (roughly  70  percent)  of  those  who  did  reading  job  postings  electronically  were 
satisfied  (see  Figure  15). 

FIGURE  14:  Online  Eeature  (Reading  Job  Postings  Electronically)  (n=514) 
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FIGURE  15:  Satisfaction  with  Reading  Job  Postings  Online  (n=179) 
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Meanwhile,  the  majority  of  the  respondents  (around  40  percent  of  «=512)  were  not  aware  of  the 
availability  of  listserv  newsletters,  but  said  they  would  have  availed  of  them  if  they  knew  of  their  existence.  Most 
(roughly  66  percent)  of  those  who  had  availed  of  these  listserv  newsletters  were  satisfied  (or  very  satisfied)  with  the 
experience  (see  Figure  16). 

FIGURE  16:  Satisfaction  with  Listserv  Newsletters  («=149) 
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Open-ended  Comments 

Lastly,  many  respondents  wrote  qualitative  comments  regarding  things  they  wanted  to  say  about  AECT’s 
services  as  a whole.  Twenty-five  responses  touched  on  the  need  for  defining  the  focus  of  AECT,  17  responses 
stressed  the  need  for  improved  communications  between  the  AECT  administration  and  membership,  81  responses 
mentioned  improving  the  conference,  61  responses  mentioned  the  need  to  improve  on-line  services,  three  responses 
suggested  active  marketing  of  AECT  to  new  members,  19  comments  expressed  happiness  with  AECT’s 
performance  and  provided  encouragement,  26  responses  mentioned  the  need  to  improve  AECT’s  leadership,  19 
suggested  improving  AECT  publications,  10  mentioned  the  need  for  better  employment  postings,  and  1 1 comments 
mentioned  the  need  for  improved  networking. 
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Conclusion 

The  553  respondents  were  largely  current  AECT  members  and  mostly  from  higher  education.  It  can  be 
seen  from  the  quantitative  data  collected  by  this  survey  that  most  respondents  were  on  the  whole  satisfied  by  the 
services  provided  by  AECT.  More  than  half  the  respondents  found  the  annual  conference  and  AECT  publications  to 
be  most  beneficial.  Within  the  conferences,  they  found  events  such  as  the  university  receptions,  Wednesday  Night 
Roundup,  and  the  division/council  receptions  to  be  most  useful  for  networking.  Most  respondents  were  also  in 
agreement  with  the  decision  to  change  the  conference  date  from  Eebruary  to  October/November.  Most  also 
considered  the  concurrently -held  NSBA  trade  show  to  be  important  to  them. 

Meanwhile,  most  of  the  respondents  were  satisfied  with  the  flagship  publications  TechTrends  and  ETR&D. 
Eurthermore,  a majority  had  visited  the  AECT  Web  site  and  successfully  logged  on  as  members.  Most  were  aware 
of  the  availability  of  the  online  membership  directory,  which  the  used  for  various  purposes.  Respondents  did  not 
seem  to  be  very  much  aware  of  the  availability  of  ETR&D  and  books  in  electronic  format,  the  facility  to  purchase 
books  online,  and  the  availability  of  listserv  newsletters.  They  were  most  aware  of  the  possibility  to  register  for  the 
conferences  and  summer  leadership  meetings  online,  of  the  ability  to  submit  and  renew  Association  memberships 
online,  and  the  ability  to  submit  conference  presentation  proposals  online.  The  respondents  were  divided  with 
respect  to  awareness  about  online  job  postings.  However,  the  vast  majority  of  those  who  had  used  any  of  the  above 
facilities  were  indeed  satisfied  with  their  experience. 

The  current  survey  results  indicate  a marked  improvement  in  AECT  services  when  compared  with  the  1999 
findings.  At  that  time,  one  of  five  respondents  indicated  that  they  perceived  “nothing”  to  be  beneficial  from  their 
AECT  membership;  and  the  electronic  presence  of  AECT  (the  Web  site)  was  literally  embarrassing  to  some 
respondents  and  perceived  as  needing  significant  improvement  by  most.  At  that  time,  AECT  was  experiencing 
financial  difficulties,  had  recently  moved  the  national  headquarters,  hired  a new  executive  director,  and  restructured 
and  streamlined  the  organization’s  governance.  The  2001  survey  results  reported  here  represent  a marked 
improvement.  AECT  has  made  significant  enhancements  to  its  Web  services,  and  members  are  using  them. 
Nonetheless,  respondents  in  2001indicated  that  the  annual  conference  can  still  be  improved  and  there  is  room  for 
further  improvement  in  electronic  services.  Overall,  there  appears  to  be  an  increase  in  member  satisfaction  with 
AECT  services  since  1999. 

The  current  survey  does  not  provide  much  evidence  from  non-members  or  from  recent  memb  ers  who  have 
not  renewed.  How  do  they  feel?  Why  are  they  not  joining  or  not  renewing?  While  2001  respondents  seemed 
happier  with  the  benefits  of  AECT  membership,  the  challenge  is  to  attract  and  retain  new  members. 
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Abstract 

The  purpose  of  this  study  was  to  evaluate  Web-based  instructional  modules  that  simulated  medical 
diagnosis.  The  modules  were  designed  by  a medical  sciences  professor  for  use  by  second-year  medical  students  in 
his  pathology  course.  Medical  students  were  subsequently  surveyed  during  their  third -year  clinical  rotations  when 
they  were  working  in  real  settings.  The  survey  instrument  was  based  on  the  5 -star  instructional  model,  most 
recently  published  as  ‘first  principles  of  instruction”  (Merrill,  2002).  We  report  the  results  of  this  survey  from  two 
cohorts  of  medical  students.  After  statistical  analysis  and  elimination  of  several  less  consistent  questions,  our  5 -star 
rating  scale  had  an  overall  internal  consistency  reliability  of  0.94.  The  medical  sciences  professor  was  not  aware  of 
Merrill' s first  principles  when  originally  developing  the  Web-based  modules.  Third-year  medical  students  surveyed 
in  this  study  agreed  that  the  modules  incorporated  first  principles  of  instruction.  Merrill  postulates  that  instruction 
is  most  effective  when  it  utilizes  first  principles. 

Background  of  this  Study 

In  September,  2001  a medical  sciences  professor  approached  this  team  of  researchers  with  a request  to 
assist  in  the  evaluation  of  Web-based  modules  that  he  had  designed  and  been  using  in  his  second -year  pathology 
course.  Although  his  class  was  small  («=25),  the  modules  represented  a large  commitment  in  terms  of  time  and 
effort.  He  was  interested  in  1)  knowing  if  the  modules  were  effective,  and  2)  having  an  evaluative  tool  designed  for 
long-term  use.  The  modules  are  publicly  available  at  http://bl-msci-007c.ads.iu.edu/c602web/602/start.htm 

The  modules  have  been  in  use  for  the  past  two  years  in  his  class.  They  were  designed  as  supplements  to 
create  “real-life”  situations  for  students  where  diagnostic  skills  could  be  practiced.  As  the  modules  unfolded, 
students  were  required  to  make  decisions  based  on  initial  perceptions,  questioning  of  patients,  results  of  tests  that 
could  be  administered,  etc.  After  each  module,  students  were  required  to  take  a quiz,  the  score  of  which  counted 
toward  their  grade  in  the  class. 

The  pathology  course  is  part  of  a regional  medical  school  program.  Medical  students  enroll  in  their  first 
two  years  at  one  of  nine  local  campuses.  The  students  whom  we  surveyed  had  used  the  online  pathology  modules 
during  their  second  year  at  a regional  campus,  but  were  contacted  during  the  third  year  after  having  completed  two 
or  three  clinical  rotations.  Rotations  occur  at  hospitals  and  other  supervised  clinical  settings  where  students  are 
required  to  make  diagnostic  decisions  that  affect  the  lives  of  real  people. 

As  this  team  of  researchers  sought  a tool  with  which  to  evaluate  these  modules,  David  Merrill’s  5 -Star 
Instructional  Design  Rating  system  was  selected  as  a starting  point.  The  5-star  rating  scale  is  based  on  Merrill’s 
(2002)  first  principles  of  instruction.  The  reader  should  note  that  the  medical  professor  who  designed  the  Web- 
based  pathology  modules  was  unaware  of  Merrill’ s work  at  the  time.  The  professor  was  interested  in  evaluating  the 
effectiveness  of  his  modules.  Since  he  had  not  collected  pre-  and  posttest  achievement  data  at  the  time  the  medical 
students  used  the  pathology  modules,  it  was  not  possible  to  get  direct  measures  of  instructional  effectiveness. 
However,  if  medical  students  were  to  rate  the  modules  with  an  adapted  version  of  Merrill’ s 5 -star  rating  scale,  then 
this  would  be  a post-hoc  indicator  of  their  effectiveness. 

Literature  Review 

The  Web-based  instructional  pathology  modules  evaluated  in  this  study  present  three  areas  of  interest  to 
researchers:  1)  techniques  for  developing  medical  student  skills;  2)  the  use  of  problem-based  learning  to  improve 
medical  school  preparation;  and  3)  methods  of  evaluating  real  world  problem-based  learning  as  to  whether  it 
effectively  prepares  students  for  patient  interaction.  Much  of  the  literature  regarding  medical  school  education 
focuses  on  the  development  of  clinical  and  patient-centered  skills  (Haidet,  Dains,  Paterniti,  Chang,  Tseng  & Rogers 
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2001).  For  example,  Peterson,  Holbrook,  Hales,  Smith  and  Staker  (1992)  considered  history  taking,  the  patient 
examination  and  laboratory  results  as  the  cornerstone  of  most  diagnoses.  To  better  illustrate  the  importance  of  this 
point  Hasnain,  Bordage,  Connel  & Sinacore  (2001)  stated,  “The  great  majority  of  medical  diagnoses,  up  to  90%  in 
the  case  of  chest  pain,  for  example,  are  made  on  the  basis  of  the  history  alone.  Although  this  is  well  established,  the 
history -taking  behaviors  of  medical  students  and  residents  have  received  little  attention  as  a measure  of  diagnostic 
reasoning.”  (p.  S14)  Peterson  et  al.  (1992)  concluded  that  expert  doctors  emphasize  history  but  medical  students 
cling  to  diagnostic  tests.  As  they  gain  more  experience,  the  students  show  an  improved  use  of  history  and 
improvement  of  patient  interviewing  skills.  In  addition,  the  test  results  and  a physical  exam  provide  support  for 
decisions  made  based  on  history.  In  fact  Peterson  et  al.  reported  it  was  difficult  to  evaluate  these  three  tools 
individually  because  they  were  each  so  vital  to  a reliable  diagnosis  in  practice.  Medical  programs  have  been  using  a 
variety  of  methods  to  teach  a level  of  proficiency  in  these  three  critical  skill  areas.  Some  studies  have  investigated 
the  use  of  mentoring  relationships  (Chou,  2001),  while  others  have  engaged  students  in  simulations  (Sakowski,  Rich 
& Turner,  2001;  Zvara,  Olympio  & MacGregor,  2001).  These  studies  all  stressed  early  acquisition  of  the  techniques 
to  build  clinical  skills. 

The  use  of  problem-based  learning  (PBL)  is  well  documented  in  the  research  literature.  While  there  are 
multiple  characterizations  of  the  term  ‘problem-based  learning,’  for  the  purposes  of  this  paper  we  accept  the 
definition  put  forth  by  Albanese  and  Mitchell  (1993),  where  PBL  is  defined  as  “an  instructional  method 
characterized  by  the  use  of  patient  problems  as  a context  for  students  to  learn  problem-solving  skills  and  acquire 
knowledge  about  the  basic  and  clinical  sciences.”  (p.  55)  What  distinguishes  problem-based  learning  is  the 
presentation  of  a problem  before  students  have  learned  basic  diagnostic  concepts.  In  addition,  most  problems  do  not 
provide  students  with  all  of  the  information  necessary  to  solve  them.  They  need  to  seek  resources  and  additional 
information  as  they  move  toward  a solution.  Furthermore,  problems  that  are  presented  to  students  should  be 
compelling  and  interactive.  Faculty  instructors’  responsibilities  are  to  support  student  solutions,  not  provide  answers 
or  direction  (Albanese  & Mitchell,  1993). 

Going  beyond  the  PBL  literature,  Merrill  (2002)  has  identified  four  additional  phases  — derived  from  extant 
instructional  theories  — that  are  necessary  for  instruction  to  be  most  effective.  Together,  he  refers  to  these  five  as 
“first  principles  of  instruction.”  Learning  is  promoted  when: 

1.  Learners  are  engaged  in  solving  real-world  problems. 

2.  Existing  knowledge  is  activated  as  a foundation  for  new  knowledge. 

3.  New  knowledge  is  demonstrated  to  the  learner. 

4.  New  knowledge  is  applied  by  the  learner. 

5.  New  knowledge  is  integrated  in  the  learner’s  world.  (Merrill,  2002,  pp.  44-45) 

Methodology 

In  designing  the  assessment  tool,  we  took  each  of  the  five  principles  and  brainstormed  a series  of  ten  or 
more  questions  for  each  principle.  Questions  were  tested  for  wording  and  understanding  with  current  medical  school 
students  and  many  were  either  dis  carded  or  reworded.  The  list  of  questions  for  each  principle  included  negatively 
worded  queries  as  well  as  positive  ones.  The  list  was  narrowed  to  a final,  comprehensive  list  of  31  questions,  which 
were  then  randomly  ordered  in  the  assessment.  In  addition,  several  introductory  questions  were  placed  at  the  front  of 
the  assessment  as  a means  of  collecting  demographic  data  and  open-ended  questions  were  added  at  the  end  to  give 
participants  a venue  for  explaining  or  expounding  upon  answers. 

Two  cohorts  of  students  were  surveyed  in  the  fall  of  2001  and  2002.  Each  group  of  students  had  completed 
at  least  two  clinical  rotations;  however,  many  of  the  students  in  the  2001group  had  completed  three  at  the  time  of 
responding  to  the  survey.  The  survey  was  administered  by  means  of  an  online  survey  and  testing  tool  used  by  the 
institution.  This  tool  was  selected  because  students  had  already  taken  several  tests  and  surveys  with  the  tool  and 
were  familiar  with  it;  it  allowed  for  remote  access  from  their  diverse  clinical  assignments;  and  it  protected  the 
students’  anonymity  while  preventing  multiple  submissions  by  the  same  individual.  The  professor  who  developed 
the  modules  contacted  the  participants  by  e-mail.  They  were  given  a link  to  the  modules  and  information  regarding 
the  length  of  time  they  would  be  available. 
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Results 

Eleven  students  responded  to  the  survey  in  each  cohort,  for  a total  n = 22.  The  results  from  both  years  were 
pooled  into  one  set  of  responses.  The  medical  professor  who  designed  the  modules  was  asked  if  the  second  group  of 
students  would  have  experienced  different  modules  than  the  first,  and  he  assured  the  team  that  only  minor  changes 
had  occurred  over  the  two  years.  The  larger  n would  allow  for  a more  effective  evaluation  of  the  reliability  of  the 
questions. 

The  reliabilities  of  the  survey  scales  were  examined  by  first  looking  at  each  group  of  questions  according  to 
Merrill’s  five  principles  and  analyzing  them  using  Cronbach’s  alpha  coefficient  for  internal  consistency.  After 
running  the  test  on  aU  of  the  questions  in  each  category,  we  looked  to  see  if  the  scale  for  each  of  the  five  principles 
could  be  improved  by  removing  items  that  were  less  consistent  with  the  scale.  Consequently,  the  pool  of  questions 
was  reduced  from  31  to  22.  Confirmatory  factor  analysis  of  the  scales  was  not  possible  due  to  the  relatively  small 
sample  size.  The  resulting  scales  and  questions  are  listed  in  Table  1. 

Table  1.  5 -Star  Scales  for  Evaluating  the  Pathology  Modules  and  their  Reliabilities 


Instructional 

Principle 

Scale  Items 

Number  of 
Reliable 
Questions 

Alpha 

Value 

Problem 

-The  online  pathology  modules  were  presented  in 
the  context  of  real  world  problems. 

*The  medical  problems  posed  in  the  online 
pathology  modules  were  unrealistic. 

- The  online  pathology  modules  utilized  very 
practical  medical  situations. 

- 1 have  encountered  medical  situations  in  my 
clinical  rotations  that  were  similar  to  case(s) 
presented  in  the  online  pathology  modules. 

* I have  not  had  ANY  cases  that  resemble  the 
examples  in  the  online  pathology  modules. 

5 

.84 

Activation 

*When  I began  each  module,  I was  overwhelmed 
by  all  the  new  information,  and  didn’t  know  how 
to  begin. 

*The  online  pathology  modules  had  little 
relevance  to  what  I really  need  to  know  to  become 
a doctor 

2 

.67 

Demonstration 

-The  online  pathology  modules  showed  examples 
of  what  was  to  be  learned  rather  than  merely  give 
information  to  me. 

* The  online  pathology  modules  were  no  different 
than  reading  the  book. 

-The  graphic  images  and  video  clips  in  the  online 
pathology  modules  made  the  procedures  and 
techniques  clear  to  me. 

- 1 found  the  graphic  images  and  video  clips  in  the 
online  pathology  modules  helpful. 

-There  were  sufficient  examples  of  normal  and 
abnormal  conditions  to  make  appropriate 
diagnoses. 

5 

.73 

Application 

-Through  the  online  pathology  modules,  I had  an 
opportunity  to  practice  and  apply  the  knowledge 
and  skills  I had  just  learned. 

-The  manner  in  which  information  was  presented 

5 

.77 
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in  the  online  pathology  modules  helped  to  clarify 
misunderstandings  or  misconceptions  that  I had. 
-The  online  pathology  modules  provided  me  with 
an  opportunity  to  practice  diagnostic  decisions  in  a 
“safe”  environment. 

-By  completing  successive  pathology  modules 
throughout  the  course,  I gained  a sense  of  my 
intellectual  development. 

-The  step-by-step  design  of  the  online  pathology 
modules  allowed  me  to  apply  what  I had  been 
learning  in  class. 

Integration 

-The  online  pathology  modules  provide  techniques 
that  encouraged  me  to  integrate  the  new 
knowledge  or  skill  into  my  medical  school 
experiences. 

-The  pathology  course  provided  me  with 
opportunities  to  demonstrate  what  I learned  on  the 
online  pathology  modules. 

- Concepts  that  I learned  in  one  online  pathology 
module  could  be  used  to  help  me. 

*The  information  I gained  from  the  online 
pathology  modules  was  not  useful  to  me  after 
completing  the  pathology  course. 

- 1 have  used  concepts  that  I learned  through  the 
online  pathology  modules  in  diagnosing  patients. 

6 

.83 

Combined 
First  Principles 

22 

.94 

*Response  values  for  items  with  negative  wording  were  reversed  for  analysis. 

Reliabilities  varied  for  each  of  the  five  categories.  The  lowest  reliability  of  alpha  = 0.67  was  associated 
with  the  ‘Activation’  scale,  after  removing  all  but  two  questions.  The  highest  reliability  was  observed  for  the 
‘Problem’  scale  with  alpha  = 0.84.  When  examining  all  22  remaining  questions,  we  found  that  the  scale  as  a whole 
had  an  alpha  value  of  0.94,  a high  level  of  reliability.  Since  the  reliability  value  is  higher  when  looking  at  the  scales 
combined  than  any  of  the  individual  scales,  this  may  indicate  that  the  five-star  criteria  have  greater  reliability  when 
applied  together  rather  than  separately.  Four  of  the  scales  had  reliability  above  0.7,  which  according  to  Nunnaly 
(1978)  is  an  acceptable  level  for  Cronbach’s  alpha. 

In  addition,  we  looked  at  subject’s  average  ratings  for  each  of  the  five  principles.  The  mean  average  scale 
value  for  each  principle  was  approximately  2.0.  The  scale  used  on  the  survey  ranged  from  1 = strongly  agree,  2 = 
agree,  3 = neither  agree  nor  disagree,  4 = disagree,  to  5 = strongly  disagree.  Scoring  of  negatively  worded  questions 
was  reversed  before  analysis.  The  overall  mean  rating  of  1.86,  based  on  the  scales  created  using  all  five  first 
principles  of  instruction,  indicated  that  students  agreed  that  the  online  pathology  modules  were  effective  forms  of 
instruction.  Activation  showed  the  greatest  discrepancy  with  responses  ranging  from  ‘strongly  agree’  to  ‘strongly 
disagree.’ 
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Table  2.  Descriptive  Statistics  for  each  Scale 


First  Principle 

Minimum 

Maximum 

Mean 

Std.  Deviation 

Problem 

1.00 

2.60 

1.80 

.41 

Activation 

1.00 

4.50 

2.02 

.73 

Demonstration 

1.00 

2.40 

1.65 

.42 

Application 

1.00 

2.60 

1.97 

.38 

Integration 

1.00 

2.60 

1.92 

.43 

Combined  Eirst 
Principles 

1.10 

2.46 

1.86 

31 

Overall  we  felt  that  the  5-star  criteria  did  indeed  provide  an  excellent  framework  from  which  to  develop  scales  to 
assess  these  online  pathology  modules.  The  students  responded  positively  to  the  questions.  Information  collected 
from  the  open-ended  questions  provides  some  insights  as  to  what  aspects  and  modules  respondents  found  most 
useful  in  their  current  situation.  Of  the  responses  received,  CBC  review  was  mentioned  the  most  often  with  five 
responses.  Other  modules  that  were  mentioned  specifically  were:  myocardial  infarction,  diabetes,  blood  pathology, 
cardiac  pathology,  gallstones,  lung  cancer,  woman  with  morning  stiffness,  chest  pain,  dyspepsia,  vaginal  bleeding, 
abdominal  pain,  hematology,  leukemia  and  alcoholism.  Clearly,  for  some  reasons,  these  stood  out  in  the  minds  of 
the  respondents,  many  of  them  listing  more  than  one  module. 

Summary  and  Conclusions 

The  first  goal  of  this  study  was  to  evaluate  the  relevance  of  the  information  presented  in  the  pathology 
modules  to  students  enrolled  in  their  third-year  clinical  rotations.  As  can  be  seen  in  student  responses  to  the 
problem-oriented  criteria  for  the  first  principle,  the  pathology  modules  seem  to  have  been  effective  towards  this 
goal.  For  example,  to  the  question,  “I  have  encountered  medical  situations  in  my  clinical  rotations  that  were  similar 
to  case(s)  presented  in  the  online  pathology  modules”,  20  out  of  22  respondents  stated  they  agreed  or  strongly 
agreed  with  this  statement.  Responses  to  such  questions  support  the  usefulness  of  the  information  presented  to 
students  during  the  clinical  rotations  and  encourage  the  collection  of  longitudinal  data.  A stronger  argument  can  be 
made  using  the  average  rating  according  to  all  five  criteria  combined.  The  average  rating  of  1.86  indicates  that  the 
respondents  agreed  that  the  modules  helped  them  generate  ideas  and  apply  diagnostic  concepts  to  real  world 
situations,  and  actively  practice  skills  that  they  would  later  use  in  clinical  experiences. 

Responses  for  improving  the  modules  were  diverse,  but  there  were  two  suggestions  that  were  repeated  by 
multiple  respondents.  These  included  having  the  modules  be  more  interactive  with  more  multimedia,  and  making 
them  more  difficult.  Students  also  commented  on  choices  that  were  presented  in  the  module . For  example,  some  of 
the  choices  were  mentioned  as  being  clearly  wrong  (i.e.  letting  a patient  go  home  when  they  were  clearly  not  well). 
There  was  some  interest  expressed  in  having  more  choices  as  well  as  more  difficult  choices.  Additionally,  a student 
suggested  that  the  pathology  modules  should  incorporate  topics  from  other  classes.  This  suggestion,  if  valid,  may 
point  to  one  reason  why  activation  scored  lower  than  the  other  four  principles.  Incorporating  material  from  other 
classes  would  increase  the  amount  of  activation  of  prior  knowledge  required  of  students  for  each  case.  In  upgrading 
and  maintaining  the  modules,  this  should  be  taken  into  consideration. 

This  study  could  prove  to  be  a valuable  place  to  launch  new  studies.  First,  results  could  be  compared  with 
other  data  sources  such  as  class  rankings  and  the  USMLE  (US  Medical  License  Examination)  results  for  validity 
studies.  Secondly,  these  modules  are  beginning  to  be  used  at  other  institutions.  Comparisons  of  student  responses 
across  different  contexts  would  be  valuable  to  show  the  effectiveness  of  the  modules  removed  from  the  class  setting 
in  which  they  were  originally  used.  Einally,  additional  support  for  depth  of  the  evaluation  could  be  established  by 
pairing  this  tool  with  practical  measures  of  clinical  skills  such  as  supervisor  evaluations  of  clinical  students. 
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Abstract 

The  effects  of  a communications  protocol  in  a synchronous  on-line  chat  environment  on  the  satisfaction  of 
students  in  a Web-based  computer  science  course  was  studied.  Two  undergraduate  faculty  members  who  teach  Web- 
based  courses,  and  42  students  in  four  separate  undergraduate  computer  science  courses  participated  in  the  study. 
Students  completed  a pretest  which  measured  their  prior  experience,  current  attitudes,  and  expectations  with 
regards  to  synchronous  chats  in  a Web-based  course.  Two  classes  of  students  were  introduced  to  a communications 
protocol  by  the  instructors  which  was  used  throughout  the  course  while  in  synchronous  chat.  The  other  two  classes 
did  not  use  a protocol.  A posttest  was  administered  to  all  students  at  the  end  of  each  course  which  measured  student 
satisfaction  with  the  synchronous  chats  held  throughout  the  semester.  A communications  protocol,  in  the  context  of 
this  investigation,  is  a set  of  rules  or  guidelines  that  are  adhered  to  by  all  participants  while  engaged  in  a 
synchronous  electronic  discussion. 

This  study  answered  the  question:  What  effect  does  a communications  protocol  in  a synchronous  on-line 
chat  environment  have  on  the  satisfaction  of  students  in  a Web-based  computer  science  course?  The  study  showed 
that  the  use  of  a communications  protocol  in  on-line  synchronous  chats  had  no  effect  on  student  satisfaction  with 
Web-based  courses. 


Introduction 

The  purpose  of  this  study  was  to  improve  student  satisfaction  with  synchronous  chat  in  a Web-based 
undergraduate  computer  science  course.  The  study  examined  the  effects  of  a communications  protocol  in  a 
synchronous  on-line  chat  environment  on  the  satisfaction  of  students  in  a Web-based  computer  science  course.  This 
study  sought  to  answer  the  question:  What  effect  does  a communications  protocol  in  a synchronous  on-line  chat 
environment  have  on  the  satisfaction  of  students  in  a Web-based  computer  science  course? 

Although  there  has  been  a significant  amount  of  research  done  on  synchronous  computer- mediated 
communications,  the  literature  on  real-time,  computer-based  communication  in  instructional  settings  is  sparse 
(Murphy  & Collins,  1997).  Only  a few  studies  have  attempted  to  examine  the  use  of  communication  protocols  in 
electronic  chats,  and,  of  those,  none  have  looked  at  the  role  such  protocols  may  play  in  student  satisfaction  with 
interaction  in  Web-based  distance-learning  courses. 

Interaction  has  been  repeatedly  cited  as  an  essential  ingredient  of  the  successful  implementation  of  Web- 
based  courses  (Shotsberger,  2000).  In  a recent  study  (Donaldson  & Thomson,  1999),  interpersonal  communication 
that  includes  instructor-to-student  and  s tudent-to-student  interaction  was  regarded  as  “important”  or  “extremely 
important”  by  83.2  percent  of  the  students  taking  a Web-based  course.  With  synchronous  communication,  the 
opportunity  to  interact  with  teachers  on  a routine  basis  was  deemed  “very  effective”  or  “effective”  as  an 
instructional  tool  by  all  participants  (Shotsberger,  2000). 

Interaction  is  a key  factor  in  student’s  learning  satisfaction,  and  it  assists  in  maintaining  the  persistence  of 
distance  students  (King  & Doerfert,  1996).  A factor  that  has  been  identified  as  influencing  student  satisfaction  with 
synchronous  interaction  is  structured  dialogue  and  interaction  which  has  been  designed  into  the  course  and  includes 
both  learner-to-learner  interaction  and  instructor-to-learner  interaction.  Also,  prior  experience  with  computer- 
mediated  communication  influences  student  satisfaction  with  interaction  in  general  in  a Web-based  course  (Vrasidas 
& Stock-Mcisaac,  1999). 

The  chat  room  (an  electronic  meeting  place  where  participants  can  communicate  with  each  other  at  the 
same  time  but  in  different  places)  is  currently  the  most  likely  candidate  to  replace  the  interactivity  of  the  traditional 
classroom  when  used  in  a virtual  classroom  environment  (Williams,  1999).  Within  the  synchronous  chat 
environment,  it  is  possible  that  new  methods  of  creating  shared  systems  of  significance  may  have  implications  for 
the  academic  discourse  (Reid,  1991).  Synchronous  environments  can  sometimes  increase  the  opportunity  for 
learners  to  improve  critical  thinking,  problem  solving,  and  communication  skills  (Marjanovic,  1999).  Synchronous 
chat  environments  potentially  are  motivational  and  can  function  as  an  effective  tool  for  interactive  learning  (Sorg, 
2000).  Paulsen  (1995)  identified  several  techniques  that  can  be  used  in  a synchronous  chat  environment  which 
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include:  a)  debates,  b)  simulations  or  games,  c)  role  play,  d)  case  studies,  e)  transcript-based  assignments,  f) 
brainstorming,  g)  Delphi-like  techniques,  h)  forums,  and  i)  group  projects.  Advantages  of  chat  rooms  include  ease  of 
logistics,  convenience,  anonymity,  and  record  keeping.  Disadvantages  include  conversation  lag  time,  lack  of  person- 
to-person  interface,  deindividuation  (or  the  tendency  to  not  perceive  participants  in  chats  as  individuals,  limited 
modes  of  communication,  artificiality,  and  lack  of  flexibility  in  accommodating  different  learning  styles  (Coleman, 
Paternite,  & Sherman,  1999).  These  disadvantages  can  lead  to  a questioning  of  the  quality  of  synchronous  computer- 
mediated  communication  and  can  be  a deterrent  to  participation  (Perdue  & Valentine,  2000). 

The  use  of  chat  environments  can  result  in  greater  collaboration,  social  interaction,  and  positive 
engagement  and  can  assist  in  providing  an  effective  forum  for  immediate  feedback  and  brainstorming  tasks.  Chat 
environments,  however,  may  not  encourage  reflective  thought  (Pena-Shaff,  Martin,  & Gay,  2001).  Knowledge 
construction,  through  interaction  and  collaboration  with  peers  and  exp  erts,  is  possible  in  synchronous  chat 
environments  (Chou,  2001).  A chat  room  may  not  be  the  only  method  for  synchronous  interaction  in  virtual 
classroom  environments,  but  it  can  serve  as  an  alternative  to  or  can  augment  other  traditional  forms  of  interaction. 
Through  the  use  of  engaging  and  structured  activities,  interaction  can  be  enhanced  (Koszalka,  2002).  Further,  certain 
subject  matter  may  lend  itself  well  to  synchronous  chat  environments  (Williams,  1999).  The  disadvantage  of  not 
being  able  to  organize  a conversation  in  a synchronous  chat  environment  is  one  issue  that  can  be  addressed  by  either 
improving  upon  software  that  is  used  for  such  purposes  or  by  improving  on  the  communication  conventions  that  are 
used  while  in  the  chat  environment  (MacDonald  & Caverly,  2000). 

Learners  need  support  to  engage  in  discussion  (ask  questions,  make  comments,  lead  discussion,  etc.),  keep 
track  of  their  discussion,  and  to  organize  their  interface  when  participating  in  synchronous  electronic  discussion 
(Veerman,  Andriessen,  & Kanselaar,  2000).  There  are  different  approaches  to  interaction  management  in 
synchronous  chat  environments.  Some  approaches  include  using  techniques  similar  to  face-to-face  interactions  such 
as  introductions,  framing,  outcome  explanation,  and  group  goal  setting  for  a chat  session  (Rintel  & Pittam,  1997). 
Dialogue  structuring  is  another  approach  where  implicit  structuring  can  induce  group  discussion  by  working  through 
key  questions.  The  results  can  be  a sense  of  greater  orientation  on  the  subject  matter  and  less  of  a chance  for 
discussion  to  drift  off-topic  (Hron,  Hesse,  Cress,  & Giovis,  2000).  Other  approaches,  such  as  3-D  virtual  chat 
environments  using  chat  software  that  differs  from  traditional  text  -based  interfaces,  can  be  effective  in  overcoming 
transactional  distance  (Altun,  1998),  although  pitfalls  to  such  approaches  include  group  sizes  which  are  too  large,  a 
lack  of  familiarity  with  the  technology,  and  poor  facilitation  skills  on  the  part  of  the  instructor  (Ingram,  Hathorn,  & 
Evans,  2000) . 

Research  that  has  focused  on  communication  conventions  in  instructional  electronic  chats  shows  that 
students  recognize  a need  to  use  such  conventions  and  protocols  in  order  to  communicate  clearly  and  minimize 
misunderstandings  in  their  on-line  transactions  with  others  (Murphy  & Collins,  1997).  This  research  points  to  the 
need  for  continued  study  of  various  means  of  conducting  instructional  electronic  chats  if  these  chats  are  to  be  an 
integral  part  of  Web-based  courses. 


Method 

Participants 

The  study  was  conducted  over  one  semester  in  four  separate  Web-based  courses  of  instruction.  The  total 
subject  pool  included  62  students  who  completed  the  pretest  instrument  with  42  of  those  students  also  completing 
the  posttest  instrument  resulting  in  42  matched  subjects.  The  main  study  courses  included:  Relational  Database 
Concepts  (comparison  group)  n=15;  Advanced  C-H-  Programming  (comparison  group)  n=8;  Computer  Ethics 
(treatment  group)  n=10;  C-H-  Programming  (treatment  group)  n=9. 

Design  and  Procedure 

The  pretest  and  posttest  questionnaires  and  instructions  that  were  used  for  this  study  were  pilot-tested  at  the 
University  with  30  students  who  were  taking  a Web-based  computer  science  course.  Internal  consistency  was  high 
for  both  the  pretest  (Cronbach’s  alpha=.89)  and  the  posttest  (Cronbach’s  alpha=.85).  Pretest  questions  numbers  eight 
and  nine  (numbers  five  and  six  on  the  posttest)  were  eliminated  from  analysis  due  to  variations  in  question  wording 
from  the  pretest  to  the  posttest  that  may  have  been  confusing  to  students  participating  in  the  study. 

Prior  to  the  study,  two  instructors  were  trained  in  the  use  of  a communications  protocol  in  synchronous  chat 
environments.  Each  instructor  was  assigned  a comparison  group  course  and  a treatment  group  course.  Identical 
Web-based  pretest  surveys  were  created  for  each  of  the  four  courses  and  were  administered  between  January  21  and 
Eebruary  1,  2002.  Students  voluntarily  participated  in  the  study  by  completing  the  pretest  and  then,  at  end  of  the 
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semester,  completing  the  posttest.  An  incentive  in  the  form  of  a raffle  of  two  gift  certificates,  redeemable  at  the 
campus  bookstore,  was  used  to  increase  participation.  Privacy  and  confidentiality  were  maintained  through  the  use 
of  anonymous  tracking  of  pretest  and  posttest  responses. 

Students  in  the  treatment  group  courses  were  oriented  on  the  use  of  the  communications  protocol  for 
synchronous  chat  that  was  then  used  throughout  the  semester  for  all  synchronous  chats.  Students  in  the  comparison 
group  courses  were  not  oriented  on  the  use  of  the  communications  protocol  and  did  not  use  a protocol  for 
synchronous  chat  at  any  time  during  the  semester. 

Results 

The  first  step  in  the  analysis  was  to  determine  whether  or  not  differences  concerning  experiences  with 
Web-based  learning  existed  between  the  treatment  students  and  the  comparison  students  that  might  influence  the 


study  results.  This  was  determined  by  assessing  student  responses  to  posttest  questions  ‘a’  through  ‘c’  that  asked 
about  prior  participation  in  Web-based  courses,  participation  in  on-line  chats  during  the  semester,  and  frequency  of 
participating  in  on-line  chats  during  the  semester.  Although  members  of  the  comparison  group  were  more  apt  to  be 
first-time  students  in  a Web-based  course  than  were  treatment  students  (10  vs.  4),  this  difference  was  not  found  to  be 
statistically  significant  based  on  the  results  of  a chi-square  analysis  X^(l)  = 2.36,  p=.ll.  (see  Table  1). 

Table  1:  Crosstab  Analysis  by  Condition  for  Posttest  Item  A 

Condition 

Treatment 

Comparison 

Total 

Post  A:  Was  this  the  first  Web-based  course  Yg^ 

that  you  have  ever  taken? 

21.1% 

(4) 

43.5% 

(10) 

33.3% 

(14) 

No 

78.9% 

(15) 

56.5% 

(13) 

66.7% 

(28) 

Total 

100.0% 

(19) 

100.0% 

(23) 

100.0% 

(42) 

The  number  of  students  who  participated  in  the  synchronous  chats  was  equal  across  the  comparison  and 
treatment  groups  (see  Table  2)  which  also  proved  not  to  be  significant  X^(l)  = 2.31,  p=.  14. 

Table  2:  Crosstab  Analysis  by  Condition  for  Posttest  Item  B 

Condition 

Treatment 

Comparison 

Total 

Post  A:  Did  you  participate  in  any  on-line  Yg^ 

chats  during  the  semester? 

94.7% 

(18) 

78.3% 

(18) 

85.7% 

(36) 

No 

5.3% 

(1) 

21.7% 

(5) 

14.3% 

(6) 

Total 

100.0% 

(19) 

100.0% 

(23) 

100.0% 

(42) 

Between  the  comparison  and  treatment  group,  66.7%  of  the  students  participated  in  at  least  four  or  more 
synchronous  chats  during  the  semester.  Although  the  comparison  group  was  more  apt  to  participate  in  greater  than 
10  synchronous  chats,  this  result  proved  not  to  be  significant  X^(l)  = 16.13,  p=.24. 

Given  that  there  were  no  significant  differences  between  the  two  groups  on  these  demographic  variables, 
the  first  analysis  conducted  was  an  analysis  of  variance  (ANOVA)  comparing  the  posttest  scores  of  the  comparison 
and  treatment  students  using  their  pretest  scores  as  a covariate  (see  Table  3).  For  this  analysis,  results  were  only 
processed  for  those  students  who  indicated  that  they  had  participated  in  a Web-based  course  prior  to  the  study  as 
assessed  by  question  ‘a’  on  the  posttest  (N=28).  There  were  no  significant  differences  between  the  treatment  and 
comparison  group  on  any  of  the  seven  variables  between  the  pretest  and  posttest.  The  comparison  group  actually 
scored  higher  on  six  of  seven  items  showing  a possible  trend,  but  the  differences  were  small  and  non-significant. 

Table  3:  Analysis  of  Variance  for  Students  With  Prior  Experience  in  a Web-based  Course  (N=28) 

Treatment  Comparison 

Mean  (1-5);  Mean  (1-5);  5 Mean 

Source 5 preferred preferred ^ Square F p 
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Posttest  #1:  Overall,  how  satisfied 
were  you  with  this  Web-Based 
course? 

4.20 

(n=15) 

4.31 

(n=13) 

1,27 

.068 

.046 

.83 

Posttest  #2:  How  satisfied  were 

4.07 

(n=15) 

4.38 

(n=13) 

you  with  the  quality  of  the 
interaction  in  the  class? 

1,27 

.011 

.020 

.89 

Posttest  #3:  How  satisfied  were 
you  with  the  interaction  that 

4.07 

4.62 

1,26 

1.69 

1.40 

.25 

occurred  between  yourself  and  the 
instructor? 

(n=14) 

(n=13) 

Posttest  #4:  How  satisfied  we  re 
you  with  the  interaction  that 

3.33 

4.00 

1,27 

1.50 

1.67 

.21 

occurred  between  yourself  and 
other  students  in  the  class? 

(n=15) 

(n=13) 

Posttest#?:  Synchronous 
interaction  (at  the  same  time,  e.g.. 

3.67 

3.62 

1,27 

.616 

1.09 

.31 

chat  rooms)  in  a Web-based  course 
is  important. 

(n=15) 

(n=13) 

Posttest  #8:  How  satisfied  were 

you  with  the  synchronous  (at  the 
same  time,  e.g.,  chat  rooms) 
interaction  in  the  course? 

3.53 

(n=15) 

3.77 

(n=13) 

1,27 

.011 

.015 

.91 

Posttest  #9:  Overall,  would  you 
say  that  synchronous  chat  sessions 
contributed  positively  to  your 
Web-based  learning  experience? 

3.33 

(n=15) 

3.77 

(n=13) 

1,27 

.473 

.664 

.42 

The  same  analysis  was  repeated  (see  Table  4)  for  all  students  in  both  the  comparison  and  the  treatment 
group  regardless  of  past  experience  taking  a Web-based  course  (N=42).  Again,  there  were  no  significant  differences 
between  the  treatment  and  comparison  group  on  any  of  the  seven  variables  at  posttest  using  pretest  scores  as  a 
covariate. 


Table  4:  Analysis  of  Variance  for  All  Students  (N=42) 


Treatment 

Comparison 

Mean  (1-5); 

Mean  (1-5);  5 

Mean 

Source 

5 preferred 

preferred 

df 

Square 

F 

P 

Posttest  #1:  Overall,  how  satisfied 
were  you  with  this  Web-Based 
course? 

4.26 

(n=19) 

4.09 

(n=23) 

1,41 

.403 

.285 

.60 

Posttest  #2:  How  satisfied  were 

4.11 

(n=19) 

4.13 

(n=23) 

you  with  the  quality  of  the 
interaction  in  the  class? 

1,41 

mi 

.031 

.86 

Posttest  #3:  How  satisfied  were 
you  with  the  interaction  that 

4.06 

4.65 

1,40 

3.63 

3.53 

.07 

occurred  between  yourself  and  the 
instructor? 

(n=18) 

(n=23) 

Posttes  t #4:  How  satisfied  were 
you  with  the  interaction  that 

3.42 

3.61 

1,41 

.069 

.074 

.79 

occurred  between  yourself  and 
other  students  in  the  class? 

(n=19) 

(n=23) 

Posttest#?:  Synchronous 
interaction  (at  the  same  time,  e.g.. 

3.68 

3.74 

1,41 

.421 

.701 

.41 

chat  rooms)  in  a Web-based  course 
is  important. 

(n=19) 

(n=23) 

Posttest  #8:  How  satisfied  were 

3.68 

3.70 

1,41 

.347 

.385 

.54 
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you  with  the  synchronous  (at  the 
same  time,  e.g.,  chat  rooms) 
interaction  in  the  course? 

(n=19) 

(n=23) 

Posttest  #9:  Overall,  would  you 
say  that  synchronous  chat  sessions 

3.53 

3.78 

1,41 

.211 

.304  .59 

contributed  positively  to  your 
Web-based  learning  experience? 

(n=19) 

(n=23) 

Paired  t -tests  were  also  performed  to  measure  pretest  to  posttest  change  for  both  the  treatment  and 
comparison  group.  The  results  of  these  tests  show  that  there  were  no  significant  changes  from  the  pretest  to  posttest 
for  either  group. 

Discussion 

The  results  of  this  study  indicated  that  the  use  of  a communications  protocol  in  synchronous  chat  did  not 
have  any  effect  on  student  satisfaction  with  Web-based  courses.  The  result  was  similar  regardless  of  whether 
students  had  prior  experience  taking  a Web-based  course  or  not.  Also,  students  did  not  have  any  significant  gain  in 
scores  from  their  pretest  to  posttest  responses  . 

Based  on  this  single  study,  it  is  difficult  to  determine  exactly  why  the  use  of  a communications  protocol  in 
synchronous  chat  had  no  effect  on  student  satisfaction.  In  prior  surveys  conducted  at  the  University,  students  have 
indicated  that  a contributing  factor  to  feelings  of  dissatisfaction  was  the  lack  of  organization  and  the  general  chaotic 
nature  of  synchronous  chats.  Based  on  the  results  of  this  study,  the  type  of  synchronous  chat  may  also  need  to  be 
examined  as  a contributing  factor  to  student  dissatisfaction. 

One  of  the  strengths  of  this  study  was  that  it  utilized  an  empirical  approach  which  employed  a comparison 
and  treatment  group.  Another  strength  of  the  study  was  that  it  used  instruments  that  were  shown  to  have  high 
reliability.  Although  the  sample  sizes  were  small,  pretest  to  posttest  mean  scores  indicated  no  trends  in  the  data 
between  the  comparison  and  treatment  groups  which  suggests  that  even  if  larger  sample  sizes  were  used,  the  results 
would  not  have  changed. 

A weakness  of  the  study  was  that  the  questionnaire  designs  (the  pretest  and  posttest)  were  more  appropriate 
for  students  with  prior  experience  with  Web-based  courses.  The  pretest  questionnaire  could  have  been  designed 
using  questions  that  better  differentiated  between  experienced  and  non-experienced  students  and  provided  questions 
that  specifically  focused  on  this  difference  among  students.  Another  weakness  is  that  all  of  the  questions  in  both  the 
pretest  and  posttest  were  close-ended.  The  addition  of  some  open-ended  questions  would  have  offered  participants 
an  opportunity  to  further  expand  on  or  to  provide  clarification  regarding  their  responses.  This  qualitative  data  may 
have  illuminated  the  question  of  why  the  use  of  a communications  protocol  had  no  effect. 

A limitation  of  the  study  is  the  fact  that  of  the  42  students  who  participated  in  the  study,  only  27  students 
actually  had  prior  experience  with  Web-based  courses.  Also,  the  study  was  conducted  with  students  in  a single 
university  setting.  Another  limitation  is  that  all  of  the  students  who  participated  in  the  study  were  taking  one  or  more 
computer  science  courses,  thus  their  knowledge  of  and  ability  to  apply  technology  may  have  been  more 
sophisticated  than  that  of  other  students  who  may  not  have  a technical  background. 

Implications 

Implications  for  faculty  who  may  or  may  not  use  a communications  protocol  in  synchronous  chat  are 
inconclusive.  Currently,  most  instructors  at  the  University  use  chat  as  a form  of  virtual  office-hour  whereby  students 
can,  at  their  option,  attend  a weekly  chat  to  ask  questions,  make  comments,  or  observe  other  student -to-student  or 
student-to-instructor  interactions.  As  discussed  in  the  following  section,  the  study  of  the  effect  of  a communications 
protocols  in  synchronous  chat  is  a suitable  area  for  continued  research  due  to  a lack  of  conclusive  research  in  this 
area. 

Recommendations 

Recommendations  for  further  research  include  exploring  other  Web-based  applications  of  chat  in  addition 
to  virtual  office-hours  such  as  instruction,  lecture,  and  social  interaction  (while  using  a communications  protocol). 
Such  research  may  contribute  to  a better  understanding  of  how  communications  protocols  impact  the  synchronous 
chat  environment. 

Faculty  at  the  University  who  currently  use  a communications  protocol  while  conducting  their  synchronous 
chats  should  continue  to  do  so.  Faculty  who  currently  do  not  use  a communications  protocol  may  still  want  to 
consider  using  one;  however,  the  potential  benefits  are  inconclusive.  A major  recommendation  for  further  study 
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includes  examining  the  differences  in  satisfaction  between  the  various  types  of  synchronous  chats  (instruction, 
lecture,  office -hour,  and  social  interaction)  when  a communications  protocol  is  used. 
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Abstract 

The  purpose  of  this  study  was  to  conduct  a formative  evaluation  of  a computer-based  simulation  program 
that  specifically  focused  on  the  motivational  effectiveness  of  a simulation  and  to  link  motivational  effectiveness  with 
various  instructional  simulation  components,  the  primary  focus  of  the  evaluation  was  to  find  out  how  the  simulation 
supports  learning  motivation,  the  Instructional  Materials  motivational  Survey  (IMMS)  (Keller, 1993}  was  applied  as 
the  measuring  instrument  for  the  formative  evaluation.  Based  on  ARCS  (Attention,  Relevance,  Confidence, 
Satisfaction)  motivational  design  model  (Keller,  1987b)  the  IMMS  was  modified  to  accommodate  specific  needs  of 
the  evaluation.  From  the  data  analysis,  instructional  simulation  components  were  identified  and  recommendations 
were  made  to  improve  motivational  effectiveness  of  the  simulation.  The  relationship  between  components  inherent 
with  instructional  simulations  and  learning  motivation  is  also  discussed. 

Introduction 

Instructional  Simulations — What  and  Why 

Instructional  simulations  provide  a learning  context  that  provides  opportunities  enabling  learners  to 
perform  acquired  skills  or  knowledge  under  a friendly  environment.  The  modern  era  of  instructional  simulation 
began  in  the  1950’ s as  a combination  of  war  gaming,  computer  science  and  operations  researches  (Crookall  & 

Wolfe,  1998).  Today  instructional  simulations  are  widely  applied  in  business  and  management  education  as 
effective  instmctional  tools  in  facilitating  learning  processes  (Faria,  1998). 

Instructional  simulations  are  known  for  their  effectiveness  in  promoting  active  learning  (Rosen0m  & 
Kofoed,  1998;  Wolfe  & Crookall,  1998)  rather  than  passive  learning  activities.  Learners  are  able  to  engage  peers 
interactively  for  intellectual  collaboration  (Spector,  2000)  and  work  with  colleagues  on  challenging  tasks  to  gain 
desirable  and  persisting  improvement  in  understanding  (Resnick,  Levine  & Teasley,  1991;  Scott,  Cole  & Engel, 
1992).  Using  instructional  simulations  to  teach  as  means  of  experiential  learning  also  enables  learners  to  be  involved 
in  the  learning  process  more  thoroughly  while  learning  motivation  is  increased  due  to  interactive  instructional 
approach  (Ivy  and  McQuillen,  1986). 

Instructional  Simulation  Development  Issues 

Although  the  instructional  simulations  of  past  decades  were  widely  accepted  in  educational  settings, 
researchers  and  educators  are  still  striving  to  formulate  this  instructional  method  into  an  applicable  science  (Wolfe& 
Crookall,  1998).  Ruben  (1999)  was  concerned  that  the  rush  to  embrace  instructional  simulation  as  the  innovative 
instructional  method  can  mismatch  necessary  developmental  evaluation  with  regard  to  design  and  application,  and 
specifically  to  issues  of  validity,  reliability,  and  utility.  In  considering  computer-based  instructional  simulations, 
Gredler  (1992)  suggested  the  complexity  of  development  should  be  considered  during  the  developmental  processes 
by  implementing  effective  evaluations. 

Learning  Motivation,  Simulations,  and  Instructional  Evaluation 

Instructional  simulations  are  well  recognized  by  their  ability  to  motivate  learners  (Ivy  & McQuillen,  1986; 
Bum  & Gentry,  1998).  However  there  is  no  sufficient  knowledge  as  to  understand  exactly  how  simulations 
influence  learning  motivation  (Yaconich,  Cannon  & Ternan,  1997).  In  order  to  gain  a better  understanding,  this 
study  uses  Keller’s  ARCS  motivational  design  model  (1987b)  to  investigate  the  relationship  between  simulations 
and  motivation.  Due  to  the  importance  of  motivation  when  it  comes  to  initiate  and  support  learning  behaviors, 
motivational  components  should  be  thoroughly  assessed  and  evaluated.  However  instructional  evaluation  often 
neglects  the  motivational  components  because  of  its  inherent  complexity  (Armstrong,  1989;  Baird  & White,  1982; 
Lee,  1990;  Merrill,  1980).  ARCS  Motivational  Design  Model  and  IMMS 
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Keller’s  ARCS  motivational  design  model  (Keller,  1987b)  prescribes  a practical  approach  in  designing 
motivational  instruction,  which  synthesized  various  inputs  that  have  already  been  discussed  and  suggest  strategies 
for  instruction  that  motivation  theories  apply  (Driscoll,  2000).  The  ARCS  motivational  design  model  (Keller,  1987b) 
was  not  initially  developed  for  computer-based  instructional  design.  Studies  proved  that  it  can  be  modified  to 
accommodate  the  applications  on  computer-mediated  or  web-based  instructional  developmental  evaluation  (Keller 
and  Suzuki, 1988;  Keller  and  Song,  1999).  Keller  (1993)  then  utilized  an  instrument.  Instructional  Materials 
Motivation  Survey  (IMMS),  as  a situational  measure  of  students’  motivational  reactions  to  instructional  materials 
based  on  the  ARCS  motivational  design  model  (Keller,  1987b)  for  broader  applications. 


Program  Description 

The  US  Army  Recruiting  Simulation  (USAREC)  was  initially  developed  with  the  intention  to  experiment 
with  innovative  economic  theories.  Currently  USAREC  is  designated  as  a diagnostic  and  planning  simulation 
(Gredler,  1992)  for  the  US  Army  as  a training  tool.  The  design  and  development  of  this  tool  was  not  based 
specifically  on  any  particular  instructional  design  model. 

The  simulation  characteristics  include  the  following  items. 

• Browser-based  multimedia  simulation  tool 

• Deliverable  through  secured  network  due  to  sensitivity  of  the  information  involved 

• Multiple  teams  can  operate  the  simulation  simultaneously  and  further  to  compete  with  each  other 

• Learning  facilitation  supported  by  both  artificial  agents  (databases)  and  human  agents 
(participants) 

• Modulated  components  enable  program  designers  to  modify  the  simulation  efficiently  according 
to  the  instructional  subjects  or  objectives 

Data  Collection  Methods 

In  order  to  investigate  motivational  support  provided  from  the  simulation  and  how  the  motivational 
evaluation  can  be  effectively  conducted,  the  IMMS  developed  by  Keller  (1993)  was  employed  as  the  measuring 
instrument  on  learners  reaction.  The  study  questionnaire  included  36  rating  questions  corresponding  to  each 
component  of  ARCS  Motivational  Design  Model  (Keller,  1987).  Qualitative  data  were  also  collected  from  the  short 
answers  questions  included  in  the  questionnaire. 

There  were  12  participants  surveyed  in  the  study.  Participants  were  randomly  selected  from  doctoral 
programs,  MIS  program  and  MBA  programs  in  the  Krannert  School  of  Management. 

Quantitative  data  is  the  main  component  from  the  measuring  instrument  in  accordance  with  Keller’s  IMMS 
(Keller,  1987).  Each  rating  question  allows  participants  to  rate  their  initial  reaction  from  1 to  5 (“Not  True’’  to  “Very 
True”).  Average  rating  points  of  each  category  of  the  ARCS  model  is  calculated  in  order  to  illustrate  the  general 
reaction  towards  each  category. 

In  order  to  collect  the  qualitative  data,  an  interview  with  the  program  author  was  conducted  to  gather  background 
information  and  further  identify  the  underlying  developmental  principles  and  instructional  objectives.  The  open- 
ended  questions  attached  to  the  IMMS  questionnaire  were  the  main  source  for  the  student’s  qualitative  responses. 

In  order  to  further  analyze  the  quantitative  data,  we  develop  a coding  system  based  on  elements  of 
designing  and  developing  computer-based  instructional  program.  Each  code  and  its  associated  implication  is  shown 
in  Table  1.  By  coding  the  qualitative  data,  we  categorized  learners’  reactions  corresponding  to  each  ARCS  model 
component.  During  the  coding  process,  if  a qualitative  response  reflected  multiple  codes,  then  the  response  was 
coded  for  multiple  codes. 
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Table  1.  Instructional  coding  system  with  associated  implication 

Code Implication 

Interface  This  code  represents  responses  regarding  interface  design  and  the  navigation  system  design 

seen  in  the  simulation. 

Content  This  code  represents  responses  regarding  content  presentation,  logic  of  layout,  and  established 

relevance  between  the  simulation  and  participants’  experiences. 

Learning  Support  This  code  represents  responses  associated  with  learning  process,  customizability  of  the 

simulation,  instructional  support  provided  from  the  simulation,  and  general  learning-  related 
reactions. 


Implementation  This  code  represents  responses  regarding  issues  associated  with  the  infrastructure  that  supports 
or  jeopardizes  the  implementation  of  the  simulation,  such  as  unexpected  server  problems.  This 
code  focuses  on  the  implementation  of  the  program  that  includes  capabilities  of  personnel, 
technology,  overall  managerial  issues,  and  the  design  of  implementation  procedures. 


Results 

Quantitative  Data 

On  a five-point  scale,  the  average  rating  for  the  12  participants  was  3.86  for  Attention,  4.01  for  Relevance, 
4.06  for  Confidence,  and  3.60  for  Satisfaction.  The  Satisfaction  component  obtained  the  lowest  rating  of  all  four 
ARCS  components.  Confidence  rating  is  the  highest  among  aU  four  components.  The  simulation  has  a moderate 
level  of  support  in  stimulating  Attention  and  establishing  Relevance. 

The  average  rating  for  the  Attention  component  of  the  ARCS  model  was  3.86  out  of  5.0.  The  simulation’s 
interface  design  is  the  main  simulation  feature  that  was  attractive  to  participants.  One  inherent  interface  factor  that 
might  have  an  impact  on  higher  levels  of  Attention  was  the  variability  of  multiple  participant  interactions.  The 
unpredictability  of  how  other  participants  would  interact  during  a simulation  session  might  contribute  to  stimulate 
participants’  curiosity  during  the  learning  process. 

The  average  rating  for  the  Relevance  component  was  4.01.  Participants  consider  the  concepts,  technical 
application,  and  logical  content  layout  effectively  establish  relevance  with  their  previous  experience  and  current  and 
future  research  interests. 

The  Confidence  component  obtained  the  highest  average  rating  of4.06.  Participants  perceive  the  possibility 
of  gaining  success  by  actively  engaging  with  the  simulation.  User-friendly  navigation  built  in  the  USAREC  also 
enables  participants  to  go  through  the  simulation  without  feeling  frustrated. 

The  Satisfaction  component  had  the  lowest  rating  3.60,  as  compared  to  the  other  three  components.  The 
unfinished  session  caused  by  technical  difficulties  failed  to  satisfy  participants’  seeming  expectation. 

Qualitative  Data 

To  make  sense  of  the  qualitative  responses  and  how  they  are  linked  with  the  ARCS  components,  the 
numbers  and  percentage  of  qualitative  responses  from  each  ARCS  component  were  categorized  by  proposed  codes 
(see  Table  2). 


Table  2.  Instructional  codes  compared  across  ARCS  components  (frequency  and  percent) 


Participant 

Responses 

Attention 

n=15 

Relevance 

n=8 

Confidence 

n=10 

Satisfaction 

n=7 

Frequency 

% 

Frequency 

% 

Frequency 

% 

Frequency 

% 

Interface 

11 

73 

3 

37 

5 

50 

0 

0 

Content 

5 

45 

8 

100 

7 

70 

4 

57 

Learning  Support 

2 

18 

0 

0 

2 

20 

5 

71 

Implementation 

2 

18 

1 

12 

3 

30 

5 

71 
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As  shown  in  Table  2,  73%  of  participants’  responses  from  the  Attention  associate  with  the  Interface  design 
issues  of  the  simulation.  All  responses  from  the  Relevance  are  coded  by  issues  related  to  content,  which  indicated 
the  importance  of  program  content  while  intending  to  establish  relevance.  The  content  is  also  relatively  influential  in 
helping  participants  feel  confident  in  finishing  the  program. 

Learning  support  and  Implementation  of  the  simulation  are  identified  as  major  elements  to  influence 
perceived  satisfaction  from  participants  toward  the  simulation. 

The  questionnaire  asked  specific  questions  related  to  ARCS.  For  the  Attention  component,  there  were  15 
responses.  We  then  coded  the  responses  using  the  four  codes.  For  the  15  Attention  responses,  1 1 talked  about 
design,  5 mentioned  Content.  Among  the  total  eight  responses  from  Relevance  component,  three  addressed  the 
Interface  design  of  the  simulation  while  all  of  them  were  associated  with  the  Content.  Seven  out  of  ten  responses  on 
Confidence  component  of  ARCS  were  related  to  the  Content  while  majority  of  responses  on  Satisfaction  component 
are  associated  with  Learning  Support  and  Implementation  of  the  simulation. 


Table  3.  ARCS  components  compared  across  instructional  codes  (frequency  and  percent) 


Interface 

n=I9 

Content 

n=24 

Learning  Support 
n=9 

Implementation 

n=10 

Frequency 

% 

Frequency 

% 

Frequency 

% 

Frequency  % 

Attention 

II 

58 

5 

21 

2 

22 

1 10 

Relevance 

3 

16 

8 

33 

0 

0 

1 10 

Confidence 

5 

26 

7 

29 

2 

22 

3 30 

Satisfaction 

0 

0 

4 

17 

5 

56 

5 50 

In  order  to  cross-examine  the  relationship  among  ARCS  components  and  the  coding  system,  the  qualitative 
data  based  on  given  codes  is  rearranged  in  Table  3.  More  than  half  (58%)  of  Interface -coded  responses  are 
associated  with  the  Attention  component.  This  might  indicate  that  instructional  interventions  associated  with 
Interface  issues  may  influence  the  attention  component  of  motivation. 

Among  the  24  Content-coded  responses,  33%  are  from  the  Relevance  and  29%  are  associated  with  the 
Confidence.  These  results  indicate  that  the  program  content  may  have  an  influential  effect  to  establish  relevance  and 
participants’  confidence  related  to  motivation,  which  also  supports  the  finding  from  Table  2.  Both  learning  support 
and  program  impbmentation  are  influential  in  making  participants  feel  satisfied.  Learning  Support -coded  and 
Implementation-coded  responses  tend  to  relate  closer  to  Confidence  and  Satisfaction  components  than  to  Attention 
and  Relevance. 

As  indicated  in  the  qualitative  data  results,  participants’  responses  tend  to  focus  on:  the  design  related 
issues  when  reporting  on  the  Attention  component  of  the  simulation;  the  content  related  issues  of  the  simulation 
when  reporting  on  the  Relevance  and  Confidence  components;  and  learning  support  and  implementation  related 
issues  when  reporting  on  the  Satisfaction  component  of  the  simulation. 

Regardless  of  the  ARCS  component  the  qualitative  data  is  associated  with,  we  see  a tendency  that 
participants  for  this  study  responded  to  learning  mo  tivation  questions  by  evaluating  the  overall  simulation  primarily 
for  its  content  and  interface  design  perspective. 


Table  4.  Coded  responses  versus  total  responses 


Total  Responses 
per  Code 

Total  Responses 

Percentage 

Interface 

19 

40 

47.5  % 

Content 

24 

40 

60  % 

Learning  Support 

9 

40 

22.5  % 

Implementation 

10 

40 

25  % 
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Discussion 


General  Motivational  Support 

In  general,  participants  rated  the  simulation  as  instructional  software  that  stimulates  attention,  establishes 
relevance,  and  builds  confidence.  The  rating  for  the  Satisfaction  component  of  the  ARCS  model  was  noticeably 
lower  than  others.  Qualitative  responses  indicated  that  the  students  didn’t  have  opportunities  to  complete  the 
simulation  due  to  experienced  technical  difficulties.  During  the  simulation  session,  the  server  responsible  for  data 
processing  was  acting  more  slowly  than  usual,  which  drastically  decreased  participants’  satisfaction  level  toward  the 
program  and  influenced  learning  motivation. 

As  indicated  by  Keller  (1987b),  in  order  to  achieve  desired  motivational  outcome  on  the  Satisfaction 
component  of  ARCS  model,  meaningful  opportunities  should  be  provided  to  participants  to  allow  them  exercise 
newly  acquired  skills  or  knowledge.  USAREC  was  designed  to  provide  meaningful  opportunities  to  participants 
however  the  infrastructure  supporting  dissemination  of  the  program  failed  to  accommodate  this  need.  Since 
USAREC  is  intended  to  be  accessible  simultaneously  by  multiple  users,  the  responsible  sever  needs  to  be  capable  to 
processing  immense  amounts  of  calculations  and  provide  feedback  to  participants  simultaneously.  During  the 
session  for  the  study,  participants  and  the  facilitator  encountered  several  technical  difficulties  causing  delayed 
automated  feedback  and  slowed  data  processing.  Interrupted  session  thus  caused  participants  to  feel  bored  and  led  to 
a lower  rating  on  Satisfaction  component  of  the  ARCS  model. 

Keller  (1987b)  indicated  that  Confidence  within  the  ARCS  model  could  be  constructed  by  building  a 
positive  expectation  for  success,  stimulating  learning  experiences  that  support  or  enhance  learners’  beliefs  in  their 
competencies.  A debriefing  session  was  conducted  after  the  simulation,  which  provided  participants  to  reflect  on 
their  previous  actions  and  results.  The  “actual”  learning  in  USAREC  depends  on  the  evolving  reflection  and 
discussions  occurred  during  debriefing  sessions.  The  simulation  establishes  an  environment  that  allows  learners  to 
evaluate  their  own  performance  collaboratively  with  peers  and  hence  to  construct  their  confidence. 

Participants  are  likely  to  evaluate  the  performance  of  the  simulation  by  looking  at  its  content  and  interface 
design.  The  learning  support  and  program  implementation  from  the  simulation  play  important  roles  in  stimulating 
participants’  satisfaction  toward  the  program.  Qualitative  responses  coded  as  program  implementation  also  indicated 
its  negative  influence  on  learning  motivation  caused  by  the  simulation  technical  problems. 

ARCS  Ratings  versus  Performance 

Motivation  is  important  in  learning,  but  there  is  also  an  important  effect  on  performance.  In  the  case  of  the 
simulation,  the  performance  is  the  use  of  the  simulation  thereby  impacting  learning.  Keller  (1987b)  suggested  that 
either  insufficient  or  excessive  motivation  could  lead  to  lower  performance  (see  Eigure  1). 


FIGURE  I . Relationship  between  motivation  and  performance 


The  half-inverted  U curve  (see  Figure  2)  that  shows  the  relative  placements  of  the  ARCS  model 
components  versus  predictive  performance.  The  figure  illustrates  the  result  from  the  audience  analysis  on  their  entry 
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motivational  level.  We  suggest  the  highest  level  of  performance  should  correspond  to  the  highest  rating  from  IMMS 
scale,  that  is,  five.  The  acceptable  range  of  motivational  level  might  result  in  higher  degree  of  performance.  By 
placing  measured  motivational  levels  on  the  half-inverted  U curve,  ARCS  component-specific  motivational 
objectives  can  be  identified  and  further  to  design  and  develop  corresponding  instructional  strategies.  In  our  case, 
ratings  on  Attention,  Relevance,  and  Confidence  components  were  relatively  higher  than  Satisfaction  component, 
which  helped  us  rationalize  the  lower  limit  of  acceptable  motivation  level. 


FIGURE  2.  Half-inverted  U curve — Relative  motivation  levels  of  ARCS  components  versus  performance  level 


Recommendations 

Effective  Implementation  to  Improve  Motivational  Effectiveness 

In  order  to  improve  USAREC’s  motivational  effectiveness  based  on  Keller’s  ARCS  motivational  design 
model,  the  simulation  needs  to  provide  opportunities  for  participants  to  apply  acquired  knowledge  and  skills.  The 
developmental  team  could  focus  on  issues  of  enhancing  the  simulations  stability  and  reliability.  It  is  recommended 
that  the  simulation  be  administered  in  a closed  network  that  has  independent  servers  and  connection  bandwidth.  This 
will  likely  avoid  jammed  connections  and  increase  the  server’s  calculation  speed.  Depending  on  the  criticality  of  the 
simulation’s  operations,  a backup  server  may  be  necessary  to  take  over  any  troubled  session.  Capable  technical 
support  personnel  should  be  standing  by  on  site  where  the  simulation  is  delivered  in  order  to  immediately  respond  to 
any  situations.  These  suggestions  will  further  help  to  evaluate  the  stability  of  the  system.  Once  the  system  is  stable 
and  operating  reasonably,  then  studens  will  be  able  to  apply  their  knowledge  and  skills  over  multiple  sessions 
thereby  perhaps  increasing  their  overall  satisfaction  level. 

Need  for  Systematic  and  Comprehensive  Evaluation  for  Instructional  Simulation  Development 

The  USAREC  team  did  not  implement  any  instructional  design  models  to  develop  the  program.  This 
simulation  required  an  immense  amount  of  data  processing.  The  process  of  designing  and  developing  this  simulation 
is  rather  complicated  compared  to  other  types  of  computer-based  instructional  programs.  Also  due  to  a high  turnover 
rate  of  developmental  team  members,  it  is  challenging  to  efficiently  monitor  the  simulation’s  instructional  qualities 
without  a formal  systematic  approach.  Thus  we  identify  an  urgent  need  to  deploy  comprehensive  instructional 
evaluative  means  with  an  emphasis  on  program  audience,  instructional  objectives,  and  technical  infrastructure  in 
order  to  improve  the  simulation’s  overall  instructional  and  motivational  quality.  A comprehensive  and  systematic 
evaluation  process  should  include  audience  analysis,  formative  evaluation,  summative  evaluation,  and  confirmative 
evaluation.  Particularly  the  technical  infrastructure  that  supports  the  implementation  of  the  simulation  should  be 
consistently  evaluated  with  regard  to  its  compatibility,  reliability,  stability,  and  customizability. 

Address  Importance  of  Motivational  Effectiveness  in  Evaluating  Instructional  Simulation 

Keller’s  ARCS  motivational  design  model  intends  to  be  implemented  prior  to  the  full  launch  of 
instructional  development  process.  An  audience  analysis  on  their  entry  motivation  level  toward  the  instructional 
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program  is  necessary  to  provide  simulation  designers  information  to  identify  motivational  objectives  and  select 
appropriate  instructional  strategies  to  achieve  desired  motivation  level.  Insufficient  or  excessive  motivation  level  can 
both  lead  to  unwanted  learning  outcome  (Keller,  1987).  The  instructional  simulations’  sound  features  associated 
with  active  learning,  experience-based  learning,  problem  solving,  and  high  level  of  interactivity  are  effective  to 
stimulate  learning  motivation.  However  it  is  relatively  easy  to  over-motivate  learners  and  make  them  feel 
overwhelmed  or  confused  about  not  knowing  where  to  begin  or  end.  We  recommend  the  implementation  of  a 
motivation  level  evaluation  on  the  audience  before,  during  and  after  the  development  of  instructional  simulations 
thereby  optimizing  programs’  motivational  effectiveness. 
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Abstract 

A team  of  researchers  joined  a development  company  to  create  an  interactive  server-based 
application  to  be  delivered  using  a classroom  of  wireless  handheld  devices.  This  study  examines  the 
instructional  performance  support  issues  related  to  the  use  of  wireless  network  technology  in  a classroom. 

The  development  framework  was  based  on  prior  professional  development  efforts  and  information  on 
developing  a network  in  a classroom  by  the  researchers  and  their  partner  corporation.  Specifically,  this 
exploratory  study  looks  at  the  instructional  use  and  teacher  performance  support  provided  by  the 
networked  assessment  and  discussion  tool.  The  results  of  this  study  include:  instructional  modes,  learner 
usability  and  perceptions,  teacher  performance  support,  and  instructional  implications. 

Introduction 

Networking  technologies  have  enabled  the  use  of  many  applications,  including  e-mail,  bulletin  boards,  and 
the  Web.  These  tools  and  others  associated  with  networking  have  transformed  the  way  we  interact.  As  networking 
technologies  mature,  educators  have  become  interested  in  understanding  and  integrating  these  network  technologies 
in  their  classrooms.  With  the  introduction  of  inexpensive  and  pervasive  wireless  networking,  new  instructional 
methods  are  possible  from  the  use  of  these  tools. 

Technology  capabilities  hold  considerable  promise  for  teaching  and  learning,  but  current  practices  may 
prove  insufficient  in  optimizing  available  resources  and  preparing  individuals  to  learn  in  a resource-rich 
environment.  Hill  and  Hannafin  (2000)  point  out  that  schools  and  classrooms  need  to  become  resource-intensive 
where  digital  resources  can  be  readily  generated  and  accessed  per  specific  goal  of  teacher  and/or  students. 

Along  with  encouraging  the  use  of  technologies  in  today’s  classroom,  as  with  any  educational  process, 
there  should  be  an  assessment  of  the  impact  of  these  initiatives.  The  evaluation  of  any  instructional  tool  cannot  be 
overlooked  if  continued  use  of  the  instructional  tool  is  expected  to  bring  about  an  impact  on  learning  for  students  of 
today  and  tomorrow. 

Background 

Roeing  Corporation,  in  association  with  Lafayette  Jefferson  High  School  and  Purdue  University,  developed 
an  interactive  software  application  for  the  purpose  of  increasing  student  learning.  The  software  intent  was  to 
increase  communication  between  teacher  and  students  as  well  as  among  students.  This  classroom  system  was 
designed  to  enable  a teacher  to  elicit  immediate  individual  responses  from  an  entire  class,  implement  electronic 
testing,  and  promote  creative  classroom  interactions.  The  system  uses  a wireless  network  to  connect  both  students 
and  teacher.  Each  student  uses  a PocketPC  with  a portable  keyboard  as  a means  of  communication. 

Our  overall  project  goal  was  to  develop  and  evaluate  a communication  tool  that  would  allow  students  and 
teachers  to  communicate  in  more  productive  and  creative  ways,  thereby  increasing  learning.  The  project  objectives 
were  twofold.  First,  the  development/implementation/research  team  wanted  to  design  and  develop  an  instructional 
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tool  to  support  learning.  Second,  we  wanted  to  conduct  developmental  research  to  investigate  the  support  tool  by 
studying  instructional  usability 

Tool  Design  & Development 

Design  Specifications 

In  order  to  meet  the  project  goal  of  increased  learning  through  the  use  of  a communication  tool,  the  project 
team  established  several  design  features  that  we  were  interested  in  developing.  We  wanted  the  tool  to  facilitate 
learning  by  providing  teachers  with  the  ability  to  (1)  pose  real-time  questions  to  the  class,  (2)  view  individual 
student  responses,  (3)  allow  teacher  evaluation  of  each  student’s  understanding  of  the  topic,  (4)  adminis  ter  exams 
electronically,  and  (5)  allow  interactive  classroom  collaboration. 

Three  design  features  were  established  to  provide  the  functionality  needed  to  create  the  desired  learning 
environment.  The  three  software  features  included: 

(1)  Automated  Response — teacher  can,  in  real-time,  continually  sample  the  opinions  and  cognitive 
responses  of  class  members  during  the  class 

(2)  Electronic  Testing — teacher  can  deliver  randomly  selected  closed  and  open-ended  test  items  to  a class 
of  students 

(3)  Creative  Interaction — teacher  can  collect  and  display  narrative  responses  from  a class  of  students 

The  ability  for  a teacher  to  get  immediate  individual  interactive  responses  from  students  will  allow  the 
teacher  to  identify  topics  that  warrant  additional  review  before  moving  on  to  new  material.  This  will  allow  a teacher 
to  immediately  review  troublesome  material.  This  may  also  greatly  reduce  the  time  a teacher  spends  grading  tests 
and  will  allow  more  time  for  classroom  activities. 

Software  Development 

The  system  consisted  of  a wire  less  networked  intranet  server  running  the  software  application  and  a 
PocketPC  for  each  student.  Based  on  the  design  specifications,  the  software  was  developed  so  that  teachers  can  use 
the  system  to  create  three  types  of  interactions:  a quick  question  (closed  and  open-ended  items),  a test  (multiple 
items),  and  a discussion  thread.  The  use  of  these  features  will  be  described  in  the  results  section. 

The  introduction  of  PocketPCs  into  the  classroom  instead  of  desktop  PCs  might  allow  existing  classroom 
configurations  to  implement  technology  that  would  otherwise  be  limited  to  a PC  lab  environment.  The  hands-on 
environment  might  also  spark  additional  improvements  and  enhancements  of  instructional  concepts. 

Developmental  Research  Methods 

Based  on  the  formative  nature  of  this  study,  the  research  team  determined  that  it  was  necessary  to  first 
conduct  an  exploratory  study  to  determine  the  critical  issues  associated  with  the  tool  prior  to  conducing  an 
experimental  study.  This  exploratory  study  used  surveys  and  observations  as  the  primary  data  to  determine  the 
application’s  efficiency  and  appeal  to  users.  Due  to  human  subjects  constraints,  we  were  only  able  to  utilize 
observation  and  system  tracking  data. 

This  study  explored  three  major  areas:  (1)  student  learning  and  instructional  usability  (understanding 
comprehension  of  the  connection  between  learning  and  the  software  functions);  (2)  technology  integration  (teachers’ 
skills  to  integrate  this  system  into  their  curriculum);  and  (3)  usability  of  the  interactive  testing  software  on  a 
PocketPC-based  network. 

Based  on  the  study’s  focus,  several  questions  emerge  that  guided  the  data  collection  and  data  analysis: 

1.  What  is  the  classroom  instructional  usability  of  a wireless  group  response  system  (defined  as  student 

learning  and  engagement)? 

2.  What  is  the  usability  and  feasibility  of  an  electronic  group  response  system  (defined  as  ease  of  use,  product 

efficacy,  user  frustration,  and  number  of  questions  asked)? 

3.  How  is  classroom  technology  integration  enhanced  through  the  use  of  an  electronic  group  response  system? 
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Instructional  Modes  Findings 

The  data  analysis  for  each  software  feature  resulted  in  the  identification  of  a related  instructional  mode. 
Based  on  research  observations,  instructor  perspective,  and  tracking  data,  this  study’s  findings  are  grouped  around 
these  three  instructional  modes.  The  instructor  used  the  tool  in  three  different  instructional  modes:  the  Rapid 
Assessment  Mode  (RAM),  the  Rapid  Testing  Mode  (RTM),  and  the  Open  Discussion  Mode  (ODM). 

Automated  Response — Rapid  Assessment  Mode  (RAM) 

The  rapid  assessment  mode  is  used  to  obtain  an  immediate  assessment  regarding  student  understanding. 

The  instructor  selects  or  creates  a quick  question  and  then  presents  it  to  the  students.  As  students  respond,  their 
inputs  appear  at  the  instructor’s  workstation.  If  the  instructor  wants  to  assess  students’  perceptions  or  cognitive 
thoughts,  this  mode  enables  the  instructor  to  rapidly  assess  all  of  the  students.  This  mode  allows  instructors  to 
quickly  assess  students,  but  also  to  complete  the  assessment  in  a comprehensive  manner,  i.e.,  to  obtain  desired 
inputs  from  all  of  the  students.  The  sampling  rate  is  not  limited  to  that  associated  with  spoken  language  assessments 
(such  as  air  time,  personalities,  competition). 

Quick  questions  are  presented  to  students  in  both  open-  and  closed-ended  items,  including  multiple -choice 
and  fill  in  the  blank  formats.  The  type  of  assessment  item  may  be  similar  to  that  found  in  the  RTM;  however,  the 
purpose  of  the  mode  is  different. 

RAM  Use  Summary 

During  the  Fall  2001  Semester,  the  RAM  was  used  in  a total  of  20  different  class  periods.  Biology  B2  used 
the  RAM  during  6 class  periods.  Biology  B3  used  RAM  in  5 different  class  periods.  Genetics  R2  and  R3  used  RAM 
in  4 and  5 class  periods,  respectively . 

There  were  a total  of  179  quick  questions  asked  in  the  RAM.  Table  1 shows  the  totals  per  class  period.  This 
ranged  from  2 to  a maximum  of  20  items.  Overall,  there  was  an  average  of  8.95  items  asked  per  class  that  used  the 
RAM.  On  the  average,  95.98%  responded  to  the  items  presented.  This  could  include  a correct,  incorrect,  or  even  a 
statement  like  “I  don’t  know.” 

Table  1:  RAM  Use  Summary 


Date 

ClassName 

Total  Items/Class 

Ave  % Response 

SD 

N participants 

Totals 

20 

179 

Average 

8.95 

95.98 

8.11 

15.7 

RAM — Techniques  Used 

In  RAM,  we  found  that  the  instructor  used  the  quick  questions  with  different  techniques.  These  include 
using  complete  electronic  questions  (including  Electronic  Stem,  Distracters,  Response)  and  partial  electronic 
questions  (including  Electronic  Stem,  Verbal  Distracters,  Electronic  Response  or  Verbal  Stem,  Verbal  Distracters, 
Electronic  Response).  In  each  case  for  the  RAM,  the  instructor  used  the  quick  question  to  solicit  quick,  parallel 
responses  via  the  hand-held  computer. 

Complete  Electronic  Question 

The  Complete  Electronic  Question  technique  is  when  the  instructor  uses  an  item  with  an  electronic  stem 
and  distracters.  These  items  can  be  created  prior  to  class  or  during  the  class.  As  appropriate,  the  instructor  will  then 
present  a quick  question  to  the  students  for  consideration.  We  believe  that  most  of  the  questions  used  with  this 
technique  had  been  constructed  prior  to  the  class  in  which  they  were  used. 

Partial  Electronic  Question 

The  Partial  Electronic  Question  technique  includes  both  electronic  stem  with  verbal  distracters  and  verbal 
stem  with  verbal  distracter.  These  items  are  usually  created  on  an  as -needed  basis.  The  instructor  will  think  of  a 
question  and  want  to  rapidly  assess  the  students,  so  he  will  create  a partial  question  (with  electronic  or  verbal  stem) 
and  then  submit  it  to  the  students.  This  technique  is  used  frequently  and  spontaneously  (see  Table  2). 

No  matter  which  techniques  are  used,  in  each  case  the  student  responds  using  the  electronic  tool.  In 
considering  the  RAM,  the  instructor  wants  to  rapidly  assess  as  many  students  as  possible.  Whether  the  question  is 
presented  electronically  or  verbally,  students  submit  their  responses  electronically,  thereby  providing  the  instructor 
with  a rapid  assessment. 
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Table  2:  RAM  Techniques  Calculations 


Electronic  Question  Technique 

Count 

Percentage 

Complete  Electronic  Question 

108 

60.3% 

Multiple  Choice 

55 

50.9% 

Short  Answer 

53 

59.1% 

Provided  Distracters 

103 

95.3% 

Distracters  Not  Provided 

5 

4.7% 

Partial  Electrailc  Question 

71 

39.7% 

Short  Answer 

71 

100% 

Total  Items  179 


Instructor’ s Role 

The  Rapid  Assessment  Mode  provided  teacher  performance  support,  in  the  sense  that  it  provided  the 
instructor  a rapid  diagnosis  of  how  the  students  were  learning  or  had  learned  the  material  at  a given  time,  as 
reflected  by  their  answers.  In  our  study,  the  instructor  was  able  to  be  flexible  in  his  use  of  this  mode  by  combining 
different  instructional  approaches  and  media  to  “customize”  the  learning  environment.  This  flexibility  was  also 
found  with  the  Open  Discussion  Mode  (see  section  on  ODM). 

In  one  biology  class,  for  example,  the  instructor  provided  a review,  using  slides  and  a laser  pointer.  He 
asked  the  questions  of  the  students  verbally;  however,  the  students  responded  by  using  the  hand-held.  In  this  way, 
the  instructor  was  able  to  get  a clear  picture  quickly  of  “where  the  students  were  at,”  allowing  him  to  diagnose 
problems  as  needed. 

From  the  instructor:  “We  reviewed  the  parts  of  cells  and  their  functions.  For  a good  portion  of  the  review  I 
just  sat  at  the  desk  and  pointed  to  cell  structures  on  the  slide  screen  with  a laser  pointer.  The  questions  were 
like  this:  "What  cell  structure  am  I pointing  to?”  or  "What  is  the  FUNCTION  of  the  cell  structure  that  I am 
pointing  to?"  These  were  not  questions  that  I had  typed  in  previously.  Instead,  I simply  verbalized  the 
questions  (the  "old  fashioned  way”!),  so  that  what  showed  up  on  the  students'  screens  was  simply  (for 
example):  "Q#250"  This  progressed  very  smoothly.  We  were  able  to  cover  a lot  of  review  material  in  a 
fairly  time -efficient  manner.”  [Instructor  Reflection,  Biology  Class,  November  26,  2001] 

RAM — Discussion 

Considering  all  of  the  qualitative  data,  two  strategic  themes  emerged  based  on  the  instructor’s  use  of  the 
tool:  confidence  builder  and  increased  student  engagement  and  participation. 

Theme:  Confidence  Builder 

Because  the  rapid  assessment  mode  provided  rapid  diagnosis  of  student  learning  at  a point  in  time,  the 
instructor  was  able  to  then  pinpoint  and  call  on  those  students  who  would  not  normally  respond,  perhaps  due  to  a 
lack  of  confidence. 

From  the  instructor:  “After  everyone  had  answered,  I would  call  on  one  student  to  say  the  correct  answer 
out  loud.  I found  myself,  of  course,  calling  on  a student  who  had  entered  a correct  answer.  This  allowed 
quiet,  shy,  under-confident  students  the  opportunity  to  answer  correctly  in  front  of  their  peers  when  called 
upon  by  the  teacher.  I tend  not  to  call  on  a quiet,  under-confident  kid,  especially  if  I don't  think  he  knows 
the  right  answer.  This  tool  allows  me  to  "read  their  minds”  and  call  on  the  kid  who  I KNOW  knows  the 
answer.  Possibly  a confidence  builder  for  that  type  of  kid????”  [Instructor  Reflection,  Biology,  November 
26,  2001] 

The  effective  use  of  this  mode  also  appeared  to  increase  instructor  confidence  in  moving  ahead  with  the 
planned  instruction.  After  presenting  a question  to  the  students,  the  instructor  could  rapidly  assess  whether  the 
students  were  at  the  appropriate  comprehension  level  to  proceed  to  the  next  stage  in  the  instruction. 

From  the  instructor:  “At  the  beginning  of  the  class  I gave  the  kids  multiple  choice  questions  one  at  a time 
just  to  check  for  their  retention  of  what  we  lectured  on  last  time.  Seeing  all  those  correct  answers  rolling 
in. . .gave  me  confidence  to  charge  ahead  today  with  the  pre-lab,  knowing  full  well  that  the  kids  understood 
the  basics  well  enough  for  us  to  move  on.”  [Instructor  Reflection,  Genetics,  December  13,  2001] 


190 


Theme:  Increased  Student  Engagement  and  Participation 

The  use  of  the  Rapid  Assessment  Mode  enabled  the  instructor  to  confirm  participation/student  engagement. 
The  instructor  also  used  this  mode  to  verify  if  students  are  following  along.  In  doing  so,  students  are  encouraged  to 
maintain  engagement  because  each  student  can  be  assessed  at  any  point  during  a class. 

From  the  instructor:  “. . . in  biology  classes  we  used  the  computers  in  a lecture  setting.  Throughout  the 
course  of  the  lecture  (in  both  periods)  I occasionally  sent  the  students  quick  questions  to  make  sure  they 
were  all  "with  me".  I'm  convinced  that  this  is  a good  way  to  make  sure  all  of  the  kids  are  engaged...” 
[Instructor  Reflection,  Biology  Class,  December  5,  2001] 

Further,  not  only  can  the  tool  promote  participation,  but  also  it  can  assess  participation  in  a quick  and  easy 
manner.  Again,  because  this  assessment  is  rapid,  students  are  more  motivated  to  pay  attention  and  focus  on  the  class 
activities.  Because  students  are  more  focused,  the  instructional  content  can  be  reviewed  in  a short  amount  of  time. 
From  the  instructor:  “Things  seem  to  be  going  well  with  the  project.  The  kids  seem  totally  comfortable, 
competent,  and  confident  with  the  computers.  We  have  been  doing  some  reviews  on  them  (with  quick 
questions).  I like  the  way  it  "forces"  every  student  to  become  actively  engaged.  I was  struck  Friday  with  the 
fact  that  a great  deal  of  material  can  be  reviewed  in  a relatively  short  amount  of  time.  . . . when  students  are 
engaged  in  this  kind  of  review  session  there  isn't  a lot  of  time  wasted  in  the  teacher  having  to  manage  class 
clowns  and  distracting  comments  because  their  minds  are  all  FOCUSED  - due  to  the  fact  that  they  HAVE 
to  be  engaged  in  this  sort  of  activity!”  [Instructor  Reflection,  November  3,  2001] 

Electronic  Testing — Rapid  Testing  Mode  (RTM) 

The  rapid  testing  mode  is  used  to  present  a series  of  questions  to  students.  The  teacher  thereby  obtains 
immediate  data  on  the  test  results  and  grading  so  that  same -period  feedback  is  possible.  This  mode  utilizes 
electronic  questions  like  those  used  in  the  RAM.  The  mode’s  cognitive  measuring  ability  is  parallel  to  that  of  the 
RAM.  However,  in  a typical  RAM,  a single  question  is  presented  and  sometimes  the  questions  are  informal,  i.e.,  the 
teacher  presents  the  stem  using  spoken  language.  The  RTM  is  more  formal.  This  would  be  much  like  a paper  and 
pencil  type  test.  In  the  RTM,  the  instructor  can  present  multiple  questions  to  the  users,  and  can  provide  an 
immediate  feedback  score  based  on  their  testing  performance. 

Due  to  the  limitations  of  this  study,  the  instructor  did  not  use  this  feature  enough  to  enable  us  to  present 
findings  and  results  related  to  this  feature. 

Creative  Interaction — Open  Discussion  Mode  (ODM) 

The  open  discussion  mode  is  used  to  engage  students  in  an  exchange  of  ideas  to  assess  their 
comprehension,  application  of  knowledge,  analytical  skills,  and  synthesis  abilities.  Using  the  ODM,  the  instructor 
presents  discussion  items  to  a group  of  students.  This  mode  allows  an  instructor  to  pose  an  open-ended  discussion 
topic  or  questions  to  s tudents.  The  students  discuss  the  topic  or  question  by  responding  to  the  instructor  and  each 
other  by  entering  their  ideas  and  then  submitting  them  to  be  shared  with  the  group.  All  of  the  comments  are 
presented  on  the  screen  of  the  handheld  device  in  chronological  order,  starting  with  the  most  recent  submission 
added  to  the  top  of  the  list.  The  instructor  has  the  option  of  making  student  inputs  anonymous. 

The  instructor  manages  the  discussion  by  making  submissions  or  by  spoken  language.  As  appropriate,  the 
instructor  presents  a new  or  revised  discussion  topic/question. 

ODM  Use  Summary 

Detailed  use  of  ODM  is  delineated  in  this  section.  The  data  are  complied  from  research  observations, 
system  tracking,  and  from  the  instructor’s  usage  journal.  The  following  table  presents  the  use  results  for  the  open 
discussion  mode. 

Table  3:  ODM  Results  Summary 

Total  Responses  per  Duration 

Date  % Response  * Responses  Responses  per  Person  Minute  (minutes) 


Totals 

2528 

Means 

89.70 

2.99 

7.29 

7.31 

* Percentages  of  students  responding  assume  100%  attendance 

During  the  Pall  2001  semester,  the  ODM  was  used  in  20  different  class  periods  for  a total  of  54  discussion 
questions.  Biology  B2  used  this  mode  6 times  with  a total  of  15  discussion  questions.  Biology  B3  used  this  mode  5 
times  with  a total  of  19  discussion  questions.  Genetics  R2  used  this  mode  in  4 classes  for  a total  of  10  discussions. 
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Genetics  R3  used  this  mode  during  4 classes  totaling  10  discussion  questions.  This  averages  out  to  2.75  discussion 
questions  per  instructional  class  in  which  they  were  used.  There  were  a total  of  2528  responses  with  a mean  of 
89.7%  of  the  students  responding  overall.  This  averages  out  to  2.99  discussion  responses  per  student.  On  average, 
each  discussion  submitted  7.29  responses  each  minute.  This  works  out  to  an  average  discussion  length  of  7.31 
minutes  per  discussion. 

ODM — Techniques  Used 

In  the  ODM,  the  instructor  can  ask  a question  or  present  a topic  or  question  for  students  to  discuss  and/or 
answer.  The  discussions  took  place  in  a synchronous  same -place  environment  with  the  instructor  present  to 
moderate  the  discussion.  In  this  mode,  overall  we  found  that  close  to  90%  of  the  students  participated  in  these 
discussions. 

The  instructor  presented  a variety  of  discussion  topics  from  reflection  question  about  a video,  ethical  issue, 
content  issues,  to  comprehension-type  questions.  The  comprehension  questions  are  much  like  the  quick  questions 
used  in  the  RAM,  but  in  the  ODM  all  of  the  students  can  see  what  the  other  students  are  thinking  as  well  as  respond 
to  their  peers.  On  the  whole,  these  topics  tended  to  be  more  application  or  analysis  type  questions  compared  to  the 
RAM  questions. 

Students  present  their  ideas  and  respond  to  each  other.  We  do  not  have  data  to  qualify  the  quality  of  the 
responses,  nor  have  we  analyzed  how  an  individual  student  performed  and  why  specific  types  of  comments  were 
made.  For  future  studies,  it  would  be  important  to  document  this  performance. 

Instructor’s  Role 

The  use  of  the  Open  Discussion  Mode  in  the  classroom  took  many  different  forms  specific  to  instructional 
approaches.  In  this  section  we  present  the  various  types  of  use  as  well  as  themes  that  describe  the  outcomes  of  the 
instructor  using  this  mode. 

Use:  Facilitating  Online  Discussion 

The  instmctor  would  occasionally  enter  new  questions,  as  one  might  in  a regular  discussion  setting, 
facilitating  the  discussion. 

“In  the  next  class  I repeated  the  same  lesson  plan,  but  I was  able  to  (somewhat)  better  follow  their 
discussion  on  my  screen. . .as  I followed  their  discussion  I occasionally  entered  new  discussion  questions 
that  they  then  responded  to.’’  [Instructor  Reflection,  Biology  Class,  October  17,  2001] 

Use:  Private  Feedback  to  Students 

Insightful  comments  led  to  teacher  use  of  private  feedback  via  the  handheld  device. 

“A  couple  of  students  would  submit  such  excellent,  insightful  discussions  remarks  that  I felt  the  need  to 
praise  them  for  it.  But  I wanted  to  do  it  privately.  So  I used  the  comment  to  student  function  in  which  I 
typed  a comment  specifically  to  that  particular  student.’’  [Instructor  Reflection,  Genetics  Class,  November 
19,  2001] 

Use:  Technology  Integration 

The  instructor  integrated  technology  as  appropriate,  combining  the  use  of  the  handheld  with  other 
instructional  media,  such  as  follow-up  discussions  of  movies  or  using  the  overhead  to  address  clarification 
questions. 

For  example,  the  instructor  used  the  overhead,  along  with  the  handheld,  to  provide  clarification  for 
questions,  following  lesson  review.  In  another  case,  the  instructor  integrated  discussion  of  a newspaper  article  with  a 
movie  regarding  cloning  of  humans.  Afterward,  the  students  logged  into  the  computer  to  discuss  aspects  of  the 
movie  and  how  it  related  to  cloning  in  the  “real  world:’’ 

The  instructor  describes:  First  I showed  them  Monday's  headline  article  in  the  Journal  & Courier  which 
featured  the  story.  Then  I showed  them  the  first  25  minutes  of  the  "The  6th  Day"...  After  the  movie  I had 
the  kids  get  out  their  computers  and  they  discussed  various  aspects  of  the  movie  and  how  the  movie  was 
both  like  and  UNlike  the  real  world  of  cloning.  [Instructor  Reflection,  Biology  Class,  November  30,  2001] 

Use:  “Pro’’  and  “Con’’  Discussions 

In  some  cases,  the  instructor  would  post  “for”  and  “against”  types  of  discussion  questions.  The  following 
vignette  is  illustrative  of  this  type  of  discussion  that  occurred. 
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[The  question:  “Do  you  think  it  would  be  OK  to  clone  humans?  What  are  some  reasons  FOR  and 
AGAINST?”]  Some  students  are  reading,  while  others  are  typing  in  the  handheld. 

Two  students  are  sitting  together  and  working  together.  It  is  very  quiet  in  the  classroom;  the  only  sound  is 
the  clicking  of  the  keyboard  and  stylus.  Later  in  the  class,  the  instructor  asks  another  question.  He  indicates 
to  the  researchers  that  the  students  appear  comfortable,  and  then  he  “tests”  that  assumption  by  asking  the 
students  what  they  think.  They  respond  positively  to  the  use  of  the  handheld,  to  the  point  of  not  wanting  to 
leave  the  one  question  to  go  on  to  the  next. 

Instructor  is  now  moving  students  to  the  second  question.  Instructor:  “I  want  to  hear  about  that  question.” 
[The  question:  “Should  the  Federal  government  fund  embryonic  stem  cell  research?  What  are  the  reasons 
FOR  and  AGAINST?”]  Instructor  asks  the  class,  “I  just  wanted  to  know — if  this  better  than  listening  to 
Instructor  talk?”  The  class  responds,  in  unison,  “Yes!!”  Instructor  tries  to  introduce  another  question,  but 
they  (as  a class)  want  to  continue  with  the  current  discussion:  One  student  says,  “I  want  to  do  more  on  this 
one.” 

Instructor  to  class:  “OK,  I’ll  tell  you  what — I’m  gonna  give  you  another  question.”  The  students  respond, 
variously,”  OK,  hold  on.”  “Chill  out,  buddy!”  Instructor  then  activates  the  third  question,”  Would  you 
approve  of  using  embryonic  stem  cells  from  your  embryonic  clone  to  cure  your  life-threatening  disease?” 
After  five  minutes.  Instructor  tries  to  wrap  up  the  discussion,  but  students  want  to  continue.  “Not  yet.” 
[Observation,  Biology  class,  October  29,  2001] 

Use:  Inviting  Analogies 

The  instructor  also  asked  for  student  analogies  to  enhance  the  review  sessions. 

The  instructor  describes:  When  we  were  finished  with  the  review  session,  we  then  had  a discussion.  For 
this  discussion  I listed  the  cell  parts  on  the  chalkboard.  The  discussion  question  . . . read  something  like  this: 
"Now  that  we  have  compared  the  cell  to  a factory,  let's  use  a different  analogy.  How  are  the  parts  of  a cell 
like  the  parts  of  a city?”  I told  the  kids  to  just  start  discussing  (on  the  computers)  as  many  examples  that 
they  could  think  of.  Some  of  the  students  had  great  examples  such  as  "The  lysosomes  (which  eat  up  and 
destroy  old  worn  out  cell  parts)  are  like  garbage  collectors."  "The  nucleus  is  like  city  hall."  [Instructor 
Reflection,  Biology  Class,  November  26,  2001] 

Use:  Case  Studies 

In  one  instance,  the  instructor  followed  review  sessions  by  using  a case  study. 

Today  both  biology  classes  used  the  computers  in  "discussion  mode”.  We  have  finished  our  cell  unit  and  I 
was  just  introducing  our  next  unit  (mitosis  & me  iosis). 

Then  I gave  them  a case  study,  which  told  the  story  of  a couple  who  had  difficulty  getting  pregnant.  They 
produced  several  early  embryos  in  an  in  vitro  fertilization  clinic,  and  had  them  placed  in  frozen  storage  for 
later  attempts  at  pregnancy.  While  the  embryos  were  in  storage  the  couple  went  through  a divorce,  so  the 
case  study  centered  around  the  question  of  who  should  have  custody  of  the  frozen  embryos.  I gave  them 
several  discussion  questions  that  I had  "preloaded"  during  first  hour  prep.  [Instructor  Reflection,  Biology 
Class,  December  6,  2001] 

In  another  instance,  the  instructor  used  the  case  study  questions  for  discussion.  The  instructor  participated 
in  the  discussion  and  redirected  it  as  necessary: 

The  discussion  topic  was  on  genetic  testing  of  newborns. . .To  kick  off  this  discussion  on  the  computers,  I 
gave  the  students  a case  study ..  .The  case  study  had  5 different  discussion  questions  at  the  end  of  it.  I had 
programmed  into  the  server  those  questions  before  class.  As  the  discussions  progressed,  I activated  the 
questions  one  at  a time.  I also  became  a participant  in  the  discussions  on  the  computers,  whenever  I wanted 
to  redirect  the  discussions.  [Instructor  Reflection,  Genetics  Class,  November  19,  2001] 


ODM — Discussion 

There  are  several  themes  that  emerged  from  the  data  analysis  of  the  ODM:  higher  level  thinking;  increased 
student  engagement  and  participation;  and  instructor  motivation. 

Theme:  Higher  Level  Thinking 

The  instructor  noted  that,  in  this  case,  the  students  had  fun  but,  again,  there  was  an  emphasis  on  their  ability 
to  use  higher-level  thinking  skills 
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“Excellent  discussions  resulted.  The  kids  had  a great  time  but  more  importantly,  they  had  to  do  some 
synthesizing  and  evaluating  (Hey!  Those  are  at  the  top  of  Bloom's  Taxonorry  1! !)”  [Instructor  Reflection, 
Biology  Class,  November  30,  2001] 

Theme:  Increased  Student  Engagement  and  Participation 

Increased  student  engagement  and  increased  participation  emerged  as  a common  them.  A common 
occurrence  was  that  the  students  did  not  want  to  stop  when  it  was  time  to  move  beyond  the  discussion:  Instructor 
reflects  on  discussions  for  two  different  classes. 

Both  genetics  and  biology  classes  have  had  a great  deal  of  success  in  using  the  computers  in  the  past  few 
days  for  discussions.  In  both  courses,  students  really  got  into  it.  I didn't  realize  before  that  this  technique 
may  help  teachers  to  not  only  meet  the  cognitive  needs  of  students  but  also  their  AFFECTIVE  needs.  They 
LIKE  chatting  in  this  way  on  the  computers  (they're  social  animals!). 

We  discovered  that  the  kids  will  respond  to  one  another's  comments  - just  like  a real  discussion!  We  also 
discovered  that  there  was  almost  100%  participation.  There  is  no  way  I could  get  100%  participation  in  a 
discussion  the  "old  fashioned  way"  with  nonacademic  types.  They  did  not  want  to  end  the  discussion  when 
I told  them  it  was  time  to  wrap  up.’’  [Instructor  Reflection,  Genetics  Class,  November  3,  2001] 

In  addition  to  observing  student  engagement  in  his  genetics  classes  (1 1th  and  12th  grade  students), 
instructor  also  found  that  there  was  increased  engagement  and  student  participation  in  his  biology  classes  (primarily 
9th  and  10th  grade  students). 

I gave  them  a discussion  question  that  they  responded  to  using  the  "discussion  mode".  They  enjoyed  it  very 
much  and  got  the  hang  of  it  quickly!  They  just  carried  on,  silently  typing  as  they  went.  At  the  end  of  class 
they  did  not  want  to  quit!  [Instructor  Reflection,  Biology  Class,  October  17,  2001] 

Theme:  Instructor  Motivation 

Another  theme  that  emerged,  perhaps  in  conjunction  with  the  higher  quality  discussions,  and  student 
engagement,  was  the  apparent  increased  motivation  on  the  part  of  the  instructor. 

Today  in  both  genetics  classes  I lectured  while  the  students  took  notes  in  their  study  guides.  At  one  point  I 
posed  a discussion  question  ("Why  do  you  think  older  women  are  more  likely  to  give  birth  to  babies  with 
chromosome  disorders?")  and  I had  the  students  engage  in  the  "discussion  mode"  on  their  computers.  These 
kids  picked  it  up  almost  immediately  (as  did  yesterday's  first  year  biology  students),  but  the  discussions 
today  were  of  higher  quality  as  the  students  stayed  "on  task"  with  the  topic.  It  was  exhilarating  for  me,  as 
an  educator,  to  see  them  quietly  and  enthusiastically  typing  away  and  forgetting  that  I was  even  there!  In 
both  classes  they  expressed  a strong  desire  to  keep  discussing  when  I decided  to  bring  closure  to  the 
activity!  They  LIKE  this!!  [Instructor  Reflection,  Genetics  Class,  October  18,  2001] 

Perhaps  as  a result  of  his  increased  motivation,  the  instructor  then  made  motivational  comments  to  the 
students.  In  one  class  session,  following  a lecture/presentation  on  Tay-Sachs,  the  instructor  activated  a question, 
“Would  you  consider  prenatal  diagnosis  and  abortion  of  a fetus  diagnosed  with  Tay-Sachs?’’  As  students  entered 
their  responses,  the  instructor  said  to  the  class,  “Can’t  wait  to  print  this  discussion  out  so  I can  really  get  into  it  and 
see  what  you’re  coming  up  with.’’ 


Comparative  Analysis  Results 

This  section  of  the  report  evaluates  the  basic  capabilities  and  limitations  of  the  electronic  system  by 
comparison  with  what  we  designate  “conventional  instructional  systems  and  tools.’’  Insofar  as  possible,  these 
comparative  capabilities  and  limitations  are  classified  as  either: 

• Intrinsic  (i.e.,  arising  from  characteristics  and  features  which  would  probably  be  found  in  any  use  of 
the  electronic  system)  or 

• Situation-specific  or  temporary  (i.e.,  associated  primarily  with  the  students,  the  teacher,  the  coding  that 
was  incorporated  in  the  tools,  and  the  subject  matter). 

In  listing  and  describing  intrinsic  features,  we  endeavor  to  isolate  the  impact  that  the  tool  by  its  nature 
would  likely  have  on  a wide  spectrum  of  possible  applications,  given  only  that  the  teacher  be  reasonably  competent 
and  willing  to  experiment  to  discover  how  he  or  she  can  best  use  the  tool  in  a teaching  environment.  By  listing  and 
describing  situation-specific  capabilities  and  limitations,  we  hope  to  suggest  methods  of  improving  or  optimizing 
features  that  are  subject  to  change  and  may  be  improved  by  further  work  on  the  concept. 
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Ease  of  Use 

Results  of  the  study  indicate  that  virtually  anyone  who  can  read  and  write  and  is  familiar  with  modern  data 
processing  can  understand  and  use  the  electronic  system  after  only  a few  minutes  of  instruction  and  practice.  Given 
the  current  level  of  computer  literacy  within  the  population  as  a whole  (at  least  in  the  United  States  and  other 
developed  countries),  ease  of  use  could  be  classed  as  an  intrinsic  feature  of  the  system.  After  the  first  lesson  or  two, 
classes  of  12  to  20  students  were  consistently  able  to  take  the  equipment  from  storage  and  set  it  up  in  approximately 
six  minutes.  If  the  equipment  were  set  up  before  the  class  began  (handhelds  and  keyboards  checked  out  and  setup  on 
each  student’s  desk),  probably  only  about  three  minutes  would  be  required  to  ensure  that  the  students  were  ready  to 
begin  a session.  This  includes  accessing  the  correct  web  page  for  the  class  as  well  as  logging  in  to  the  server.  (This 
assumes  that  the  hardware  is  part  of  the  classroom.  If  students  individually  owned  the  device,  this  time  might 
decrease.)  As  currently  programmed,  once  the  electronic  system  is  physically  in  place  there  is  a simple  path  to  each 
of  its  available  modes.  If  other  modes  are  desired,  a simple  pop-up  menu  would  still  suffice.  It  appears  that,  for  most 
foreseeable  applications,  its  ease  of  use  would  be  assured. 

Inasmuch  as  teachers  and  perhaps  most  students  would  already  be  familiar  with  the  uses  of  a conventional 
system,  it  is  pointless  to  compare  the  ease  of  use  of  the  two  systems.  The  key  point  is  that,  for  a new  system,  the 
electronic  system  is  impressive  in  terms  of  ease  of  use. 


Impact  on  the  Classroom  - The  Changed  Environment 

The  use  of  the  electronic  system  changes  the  classroom  experience  and  environment  in  ways  that  open 
many  opportunities  to  teachers  and  students.  Some  changes  are  obvious,  but  others  may  be  unexpected.  This 
discussion  covers  some  of  the  quantitative  and  qualitative  ways  the  use  of  the  electronic  system  changes  the  learning 
environment. 

The  instructor  used  the  open  discussion  mode  with  four  classes,  designated  Bio  Bl,  Bio  B2,  Gen  R2,  and 
Gen  R3.  Using  Bio  B3  for  10/17/01  as  an  example,  there  were  80  responses  from  the  class  over  the  electronic 
system.  The  class  size  was  20,  and  each  student  entered  at  least  one  response  to  the  question.  This  equates  to  80/20 
or  an  average  of  four  RESPONSES/PERSON.  There  was  an  elapsed  time  of  10  minutes  while  the  discussion  was 
underway,  so  the  instructor  received  an  average  of  eight  RESPONSES/MINUTE.  As  mentioned  in  a footnote  to  the 
table,  we  have  assumed  that  all  students  were  present  for  the  discussion.  If  any  were  absent  during  a discussion,  the 
overall  average  response  rate  would  be  greater  than  the  89  % appearing  at  the  bottom  of  the  table. 

The  data  in  the  table  helps  support  the  following  findings: 

• The  percentage  of  students  actively  and  voluntarily  participating  through  relevant  questions  and 
answers  during  discussion  periods  is  much  higher  when  the  electronic  system  is  in  use.  Whereas  in  the 
study  a typical  participation  rate  was  about  25  to  33%  (often  with  the  same  students  participating  in 
each  class)  when  “conventional”  teaching  methods  were  used,  the  active  participation  rate  when  the 
electronic  system  is  in  use  is  around  90  percent  and  perhaps  much  higher.  In  the  case  of  the  electronic 
system,  the  degree  of  participation  by  individual  students  also  varies  by  topic  and  from  day  to  day. 

• Even  though  use  of  the  electronic  system  leads  to  a great  increase  in  participation,  an  “electronic 
discussion”  does  not  necessarily  require  more  classroom  time  than  does  a conventional  question-and- 
answer  face-to-face  discussion.  This  is  because  the  electronic -based  discussion  is  essentially  a parallel 
discussion  whereas  a conventional  discussion  is  a “one  at  a time,  please“  sequential  operation.  Thus, 
the  total  number  of  student  responses  per  unit  time  is  much  greater  when  the  electronic  system  is  being 
used.  As  a rule  of  thumb,  the  information  transfer  rate  between  teacher  and  student  is  increased  by  a 
factor  of  about  2.5  when  the  electronic  system  is  put  in  operation. 

• In  an  electronic  discussion,  the  students  tend  to  respond  not  only  to  the  question  posed  by  the  teacher 
but  also  to  the  responses  being  submitted  by  other  students.  The  teacher  becomes  more  of  a moderator 
than  in  the  conventional  case. 

• In  the  conventional  classroom,  the  question-and-answer  sequence  tends  to  be  “bang -bang”  or  rapid 
response  by  both  teacher  and  student.  Our  qualitative  evidence  appears  to  confirm  that  most  written 
responses  are  more  considered  and  complete  than  are  the  verbal  responses  in  a conventional  classroom. 
This  has  obvious  implications  where  higher-order  thinking  is  desired.  This  appears  to  hold  true  for 
both  teacher  and  student. 

There  is  similar  data  for  use  of  the  rapid  assessment  mode.  The  student  participation  rate  (96.2%)  is  even 
higher  than  in  the  open  discussion  case  described  above. 
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Assuming  that  most  teachers  and  educational  decision-makers  would  welcome  an  environment  which 
increases  student  participation  by  a factor  of  more  than  three  and  information-transfer  rates  by  a like  factor,  we  next 
consider  issues  that  may  be  of  special  interest  to  groups  such  as  teachers,  students,  administrators,  and  technicians. 


Impact  on  Instructor  Preparation 

In  preparing  their  lessons  for  electronic -equipped  classrooms,  successful  instructors  will  need  to  plan  for  a 
significantly  different  experience  than  before.  Because  the  teacher/student  question-and-answer  interface  is 
essentially  parallel  rather  than  serial,  as  with  the  conventional  systems,  the  teacher  should  have  the  choice  of  (a) 
covering  more  material  per  class  than  before  or  (b)  covering  the  same  amount  of  material  more  thoroughly.  Because 
of  the  greater  level  of  student  participation  (especially  in  the  rapid  assessment  mode),  they  will  be  able  to  make  real- 
time changes  in  the  subject  matter  to  meet  the  emerging  needs  of  individual  students.  They  should  anticipate 
unexpected  twists  and  surprising  insights  when  in  the  discussion  mo  de,  and  they  will  find  that  they  need  to  consider 
well  in  advance  how  their  discussion  questions  should  be  phrased,  how  to  expand  upon  or  otherwise  clarify 
important  subjects  when  they  discover  the  need  in  mid -presentation,  and  how  they  can  take  advantage  of  a much 
higher  rate  of  inputs  from  the  students.  These  seem  to  be  intrinsic  features  of  presentations  based  in  large  part  on  the 
use  of  the  electronic  system.  By  making  good  use  of  the  electronic  testing  mode  of  the  electronic  system,  they  can 
expect  to  save  time  in  some  after-class  activities  and  gain  some  guidance  on  follow-up  needs. 


Impact  on  Student  Performance  and  Perceptions 

This  subject  requires  more  controlled  experimentation  to  help  enable  us  to  perform  a convincing  cost- 
benefit  evaluation  of  the  use  of  the  electronic  system.  In  the  present  discussion,  we  attempt  to  relate  the  very 
favorable  student  responses  to  the  intrinsic  features  of  the  system  as  a guide  to  further  evaluation. 

Confidence  and  Confidentiality 

In  the  current  electronic  system,  responses  can  be  either  anonymous  or  signed  in  the  discussion  mode.  Our 
observation  was  that  they  were  almost  always  signed,  but  some  students  have  remarked  that  they  found  it  much  less 
stressful  to  enter  a response  in  the  system  (even  if  signed)  than  to  express  it  aloud  before  the  entire  class.  The  impact 
of  the  higher  participation  rate  and  ratios  using  the  electronic  system  may  be  measured  directly  or  indirectly  in  the 
quantifiable  factors  mentioned  above:  learning  rate  by  type  of  learning,  attendance  rate,  and  student  satisfaction  with 
the  classroom  experience. 

“Canned”  and  Extemporaneous  Graphic 

Questions  and  answers  in  the  observed  classes  were  entirely  narrative  in  nature  as  far  as  electronic  system 
inputs  and  outputs  were  concerned.  The  teacher  always  resorted  to  other  tools  when  graphic  presentations  were 
needed.  The  students  never  used  graphics  in  their  responses,  regardless  of  the  tools  the  teacher  used.  Given  the  well- 
demonstrated  advantage  of  graphics  in  many  learning  processes,  the  potential  of  using  graphics  with  the  electronic 
system  needs  to  be  explored.  Various  simple  graphic  programs  perhaps  could  and  should  be  placed  at  the  teacher’s 
and  students’  disposal  using  the  electronic  system.  The  small  size  of  the  screen  may  be  a limiting  factor  in  this 
respect;  however,  this  perhaps  could  be  dealt  with  by  including  a single  large  screen  in  the  system  and  enabling  the 
student  to  zero-in  on  parts  of  this  screen  as  needed. 

Taking  Note  of  Taking  Notes 

An  infrequent  student  complaint  was  that  the  hand-held  computers  and  screen,  small  as  they  were,  still  took 
up  too  much  space  on  the  students’  desks.  The  students  did  not  seem  to  take  many  notes  whatever  the  tool  in  use  in 
the  classroom,  but  it  perhaps  should  be  made  possible  to  use  the  hand-held  computer  to  save  and  index  useful 
comments  input  by  themselves,  the  teacher,  or  other  students,  and  to  enter  and  save  their  own  notes  as  a presentation 
progresses. 

Technology  Integration 

The  section  above  on  Instructor’s  Role  in  the  section  on  Creative  Interaction — Open  Discussion  Mode 
(ODM)  contains  some  speculations  about  the  integration  of  added  hardware  and  software  with  the  electronic  system. 
A further  possibility  is  the  integration  of  the  electronic  system  with  teleconferencing  technology  so  that  various 
classrooms  could  interact  in  the  lesson  presentation.  This  capability,  which  might  be  used  for  special  classroom 
subjects,  might  have  the  advantages  of  enabling  students  with  more  diverse  backgrounds  and  experiences  to  enrich 
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the  discussions.  It  might  also  permit  the  use  of  subject-matter  specialists  who  could  simultaneously  share  their 
insights  with  a large  number  of  students,  each  in  a relatively  intimate  setting. 

We  tend  to  think  of  technology  in  terms  of  tools  or  as  the  outputs  of  physical  science.  This  of  course  is  not 
an  all-inclusive  view  of  technology.  Instructional  methods  will  continue  to  evolve,  and  the  electronic  system  must  be 
capable  of  accommodating  the  findings  of  instructional  science.  Additional  research  will  perhaps  enable  us  to 
predict  how  the  electronic  system  can  keep  pace  with  the  evolving  needs  of  the  classroom. 

Higher- Level  Thinking 

The  section  above  on  Instructor’s  Role  in  the  section  on  Creative  Interaction — Open  Discussion  Mode 
(ODM)  also  offered  some  subjective  evidence  that  the  electronic  system  fosters  higher-level  thinking,  especially  by 
those  who  tend  not  to  participate  actively  in  conventional  instructional  situations.  If  this  is  true,  it  becomes  one  of 
the  principal  intrinsic  advantages  offered  by  the  electronic  concept.  It  is  widely  believed  that  automation  and 
advancing  science  in  general  tend  to  work  against  unskilled  labor  and  in  favor  of  those  who  can  think  creatively  and 
adapt  to  rapidly  changing  situations.  The  flexibility  and  power  of  the  electronic  system  may  prove  able  to  improve 
education  in  general  and  stimulate  higher-level  thinking  in  particular. 

Conclusion 

The  electronic  system  creates  a new  learning  environment  to  which  the  teacher  and  at  least  most  students 
responded  favorably.  However,  there  were  instances  in  which  the  system  was  not  used  because  of  real  or  perceived 
physical  or  software  limitations.  In  most  cases,  it  was  easy  to  identify  potential  means  of  removing  these  limitations. 

The  electronic  system  has  a number  of  qualitative  advantages  over  other  commonly  used  instructional 
methods.  This  was  only  demonstrated  in  a single  setting  and  in  two  presumably  representative  high-school  courses. 
The  results  were  almost  uniformly  favorable.  Because  the  teacher  periodically  used  conventional  methods  of 
instruction  as  well  as  instruction  based  on  the  electronic  system,  it  was  possible  to  make  some  qualitative 
comparisons  with  conventional  methods.  These  were  also  favorable  for  the  electronic  system. 

These  favorable  results  appear  to  more  than  justify  further  investigation  of  the  potential  inherent  in  the  use 
of  the  electronic  system.  Quantitative  evidence  of  higher-level  thinking  would  be  a focal  point  of  such  study.  Of  the 
numerous  paths  one  could  take,  that  of  translating  the  observed  qualitative  advantages  of  the  system  into  quantitative 
results  that  could  be  used  by  educational  decision  makers  appears  to  be  most  logical  and  potentially  useful.  The 
results  of  the  work  recommended  herein  will  of  course  be  most  easily  applied  to  environments  similar  to  that  to  be 
used  in  the  follow-up  study,  but  it  should  be  possible  to  extrapolate  the  results  and  conclusions  to  numerous  other 
learning  environments. 
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Understanding  E-dropouts? 
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University  of  Georgia 


Abstract 

The  purpose  of  this  study  is  to  investigate  the  factors  that  affect  the  dropout  of  adult  learners  in  e-learning 
through  an  in-depth  literature  review  on  the  dropout  of  adult  learners  in  e-learning.  Based  on  variables  identified 
from  the  literature  review  of  dropout  in  e-learning  and  the  models  of  dropout  dealt  with  above  five  constructs  were 
categorized:  individual  background,  motivation,  academic  integration,  social  integration,  and  technological 
environment. 

Introduction:  Knowledge -Based  Economy  and  E-learning 

Our  society  is  continuously  moving  towards  a knowledge-based  economy:  an  economy  in  which  the 
application  of  knowledge  replaces  capital,  raw  materials,  and  labor  as  the  main  means  of  production.  The 
synergy  of  combining  new  information  and  communication  technologies  with  human  skills  has 
dramatically  altered  job  content  and  skills  requirements  at  the  workplace.  (The  Canadian  Vocational 
Association  and  UNEVOC-Canada,  2002) 

The  so-called  information  revolution  triggered  by  advanced  communication  technologies  such  as  the 
internet  has  had  a significant  influence  on  our  daily  lives.  The  arenas  of  education  and  training  are  no  exception.  The 
rapid  rate  of  change  demands  an  ability  to  learn  to  adjust  quickly  and  assimilate  large  amounts  of  conflicting 
information.  In  this  environment,  an  ability  to  learn  continuously  is  becoming  imperative.  The  learning  environment 
for  today's  learners  is  no  longer  set  within  the  walls  of  a school,  but  rather  is  everywhere,  especially  the  Web  and  e- 
mail.  These  advanced  information  technologies  allow  learners  to  access  a variety  of  learning  activities  beyond  the 
limitations  of  time  and  place.  Responding  to  these  potential  of  anytime,  anywhere  learning,  education  and  training 
stakeholders  have  made  considerable  technology  investments  in  recent  years  (The  Software  & Information  Industry 
Association  [TSIIA],  2001).  “Most  now  recognize  the  power  of  technology  to  transform  learning  into  more  flexible, 
personalizes  and  accountable  endeavor  required  by  today’s  knowledge-based  economy”  (TSIIA,  2001).  Whiteman 
(2001)  asserts  that  today's  workplace  environment  necessitates  knowledgeable,  flexible,  efficient,  and  adaptable 
workers  who  are  lifelong  learners.  He  continues  that  “adult  learners  need  to  be  updated  on  the  latest  changes  in  the 
structure  of  the  business  environment”  (p.  1).  E-learning  has  the  potential  to  meet  the  demand  of  today’s  business 
environment  for  skilled  workers  who  are  lifelong  learners.  E-learning  provides  many  potential  benefits  to  both 
companies  and  workers  (National  Alliance  of  Business  [NAB],  2000). 

Although  e-learning  has  some  advantages  as  an  efficient  and  effective  learning  delivery  media,  the  big 
problem  of  e-learning  is  learner  dropout.  While  e-learning  seems  to  answer  a lot  of  learner's  needs,  drop-out  rates 
are  higher  than  those  for  face-to-face  course  campus-based  learning  (Knowledgenet,  2001).  Svetcov  (2000)  claims , 
“It  is  generally  agreed  that  attrition  rates  from  online  schools  are  higher  than  from  traditional  schools  . . . the  online 
student  dropout  rate  [is]  around  35  percent,  [which  is]  15  percent  higher  than  traditional  schools. . ..The  fact  is,  much 
of  what  passes  for  online  education  today  would  put  most  of  us  to  sleep”  (p.  3).  More  skeptically,  Murphy  (2001 ) 
argues  that  e-learning  courses  without  face-to-face  classroom  training  have  low  success  rates— only  about  10  percent 
of  employees  complete  online-only  courses.  The  "anytime,  anywhere"  nature  of  at-your-laptop  learning  all  too 
easily  becomes  "no  time,  nowhere";  the  average  dropout  rate  for  online  courses  can  run  as  high  as  50  or  75  percent, 
depending  on  the  source  (Ganzel,  2000).  Although  many  studies  related  to  e-learning  have  been  conducted  in  the 
field  of  adult  education  or  HRD,  relatively  little  concern  has  been  given  to  why  adult  learners  dropout.  In  addition, 
there  is  not  any  research-based  evidence  about  how  and  why  the  learners  in  e-learning  programs  drop  out.  This  study 
will  provide  an  understanding  of  the  dropout  phenomena  of  adult  learners  in  e-learning  by  clarifying  the  dynamic 
process  that  results  in  the  dropout  of  adult  learners.  The  purpose  of  this  study  is  to  investigate  the  factors  that  affect 
the  dropout  of  adult  learners  in  e-learning  through  an  in-depth  literature  review  on  the  dropout  of  adult  learners  in  e- 
learning. 

Studies  of  the  Dropout  of  Adult  Learners  in  El-learnlng 

Although  many  studies  related  to  e-learning  have  been  conducted  in  the  field  of  adult  education  or  HRD, 
relatively  little  attention  has  been  paid  to  why  adult  learners  dropout.  There  is  no  broad-based  quantitative  study 
pointing  to  evidence  of  a widespread  dropout  problem  for  online  training  in  the  corporate  world  (Zielinski,  2000).  In 
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addition,  there  is  not  any  research-based  evidence  about  how  and  why  adult  learners  in  e-learning  programs  drop 
out.  Of  those  studies  of  dropout  of  adult  learners  in  e-learning  reviewed  in  this  section,  only  a few  provided  a 
comprehensive,  theoretically  -based,  and  explanatory  framework  from  which  to  analyze  the  problem  of  dropout. 
Opinion  papers  based  on  the  authors’  live  instructing  or  managing  experiences  of  e-learning  are  reviewed  and  are 
discussed  here  as  well  as  several  theory -based  studies,  because  of  their  relevance  to  the  conceptual  framework  and 
findings  of  this  research. 

A study,  the  Learning  Technology  Acceptance  Study:  “If  We  Build  It,  Will  They  Come?”  (2001),  by  the 
American  Society  for  Training  & Development  (ASTD)  and  The  MASIE  Center  reveals  the  fact  that  dropout  rates 
for  online  training  are  high  when  learners  are  put  off  by  one  or  more  several  factors.  These  factors  include  poor 
incentives  to  learn,  lack  of  accountability  for  completing  classes,  problems  with  technology,  and  the  inability  of 
poorly  designed  courseware  to  hold  a student’s  attention.  Based  on  its  own  experience  as  an  e-learning  provider. 
Frontline  Group  (2001)  also  provides  five  reasons  why  adult  learners  drop  out  in  e-learning  programs:  poor  design, 
failure  to  understand  the  new  medium,  not  considering  a variety  of  learning  styles,  lack  of  supporting  systems,  and 
ignoring  the  self-selecting  content  needs  of  learners. 

Based  on  studies  conducted  by  e-learning  providers  and  the  opinions  of  e-learning  experts,  Frankola 
(2001b)  argues  that  adult  learners  drop  out  in  e-learning  courses  due  to  the  following  reasons:  students  don't  have 
enough  time,  lack  of  management  oversight,  lack  of  motivation,  problems  with  technology,  lack  of  student  support, 
individual  learning  preferences,  poorly  designed  courses,  and  substandard/inexperienced  instructors.  Interestingly, 
NYUonline  found  that  “e-learners  who  took  only  the  asynchronous  course  were  much  less  likely  to  complete  it  than 
e-learners  who  also  participated  in  live  sessions”  (Frankola,  2001). 

On  the  other  hand,  crucial  interactivity  with  faculty  and  among  other  students  can  be  important  for  the 
success  of  a course.  Studies  conducted  by  Sun  Microsystems  Inc.  show  that  “only  25%  of  employees  finish  learning 
content  that's  strictly  self-paced,  but  75%  finish  when  given  similar  assignments  and  access  to  tutors  through  e-mail, 
phone  or  threaded  discussions”  (Frankola,  2001).  Hossein  Arsham,  a Wright  Distinguished  Research  Professor  of 
Statistics  and  Management  Science  at  University  of  Baltimore,  also  points  out  that  interactivity  with  students  is  a 
key  factor  in  explaining  students’  retention,  based  on  the  experience  of  teaching  two  courses  of  the  first  all-online 
accredited  Web  MBA  program  (Elearningpost,  2001). 

This  fact  is  also  supported  by  a study  (Towles  et  al.,  1993)  conducted  in  the  field  of  distance  education. 

This  study  sought  to  evaluate  the  effect  of  faculty  initiated  contacts  on  student’s  persistence  within  a large  video- 
based  distance  learning  program,  and  showed  that  faculty-initiated  efforts  seem  to  have  the  greatest  effect  on 
improving  course  persistence  among  freshmen  students.  Vrasidas  and  Mclsaac  (1999)  examined  the  nature  of 
interaction  in  an  online  course  from  both  teacher  and  student  perspectives.  They  find  that  the  structure  of  the  course, 
class  size,  feedback,  and  prior  experience  with  computer-mediated  communication  (CMC)  all  influenced  interaction. 
In  particular,  findings  showed  that  some  elements  of  structure,  such  as  required  activities,  led  to  more  interaction, 
and  students  who  were  new  to  CMC  were  not  comfortable  participating  in  the  online  discussion.  In  addition,  “when 
students  do  not  receive  feedback,  they  do  not  continue  to  post  messages.  Unless  receive  immediately  feedback,  they 
feel  they  are  posting  to  the  network  without  any  response”  (Vrasidas  & Mclsaac,  1999,  p.  33). 

According  to  Gilroy  (2001),  the  CEO  of  the  Otter  Group,  low  enrollments  and  high  attrition  rates  stem 
from  user  dissatisfaction  and  the  cause  of  this  problem  is  the  separation  of  people  in  time  and  space;  but  it  can  be 
overcome  by  building  environments  where  people  talk  to  one  another,  build  relationships,  and  teach  one  another. 

She  continues,  “While  there  is  no  simple  answer,  there  is  one  key  idea  that  has  been  overlooked  in  the  design  and 
implementation  of  many  of  the  e-learning  programs  on  the  market  today”.  That  is,  “learning  is  fundamentally  both 
social  and  experiential.  It  is  the  context  of  the  learning-all  of  the  elements  that  comprise  the  experience  around  the 
content -that  is  most  important”.  Based  on  the  Otter  Group's  model  of  how  best  to  teach  and  learn  online,  she 
presents  many  elements  that  must  be  managed  to  create  e-learning  programs;  Not  too  much  content  and  too  little 
context,  valued  learning  experience,  course  as  learning  communities,  personalizatfan,  and  an  open  technology 
source. 

A study,  “Student  support  services  and  success  factors  for  adult  on-line  learners,”  conducted  by  Greer, 
Hudson,  and  Paugh  (1998)  examined  a variety  of  student  support  services  and  four  areas  for  student  success  from 
the  viewpoint  of  World  Wide  Web-based  learners  in  the  University  of  Central  Florida  College  of  Education, 
Vocational  Education  area.  They  point  out  that  the  most  common  theme  in  terms  of  students'  perceptions  of  success 
factors  were  budgeting  time,  being  self -motivated,  and  having  supportive  friends  and  family. 

Shepherd  (2001)  argues  that  the  reason  why  learners  dropout  is  a simple  one  of  motivation.  In  addition, 
motivation  has  two  determining  factors:  the  first  factor  is  a desirable  outcome,  whether  this  is  the  achievement  of  a 
personal  goal,  recognition  from  others  or  some  form  of  tangible  reward  such  as  money  or  promotion.  There  is  a flip 
side  to  this,  in  that  the  learner  may  be  seeking  to  avoid  some  penalty,  such  as  a reprimand,  disapproval  or  some 
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financial  disincentive.  The  second  factor  in  motivation  is  the  learner’s  perception  of  the  likelihood,  given  that 
learners  put  in  sufficient  effort,  of  the  learner  obtaining  their  reward  or  avoiding  the  penalty.  If  the  means  to  the  end 
is  too  tortuous,  the  motivation  will  drop  regardless  of  how  desirable  the  outcome  may  be.  He  maintains  that  “even  if 
the  incentives  are  sufficient  to  get  learners  started,  e-learning  can  place  many  obstacles  in  the  way  of  successful 
completion.  “Removing,  or  reducing  the  effect  of  these  obstacles  is  essential  to  curing  the  drop-out  problem’’ 
(Shepherd,  2001).  These  obstacles  are  inappropriate  or  inadequate  content,  lack  of  time  and/or  inadequate  time  to 
learn,  no  support  for  their  learning  by  peers  and  training  mangers,  and  the  assessment  of  the  learner’s  learning 
process  by  tutors  or  managers. 

Based  on  a case  study,  Chyung  (2000,  2001a,  2001b)  finds  some  reasons  for  dropout  in  online  distance 
education.  These  reasons  are  discussed  under  the  Foundation  Section  (p.  14).  Chyung,  Winiecki,  and  Fenner  (1998) 
found  that  the  satisfaction  of  adult  learners  in  an  on-line  course  during  the  first  or  second  courses  was  the  major 
factor,  which  determined  learners'  decisions  about  whether  or  not  to  continue  in  the  program.  Forty-two  percent  of 
the  students  who  dropped  out  expressed  dissatisfaction  with  the  learning  environment  as  the  reason;  Another  reason 
given  was  a discrepancy  between  professional  or  personal  interests  and  course  structure.  In  a study  by  Lim  (2001)  to 
develop  a predictive  model  of  satisfaction  of  adult  learners  in  a Web-based  distance  education  course  and  their  intent 
to  participate  in  the  future,  she  found  that  computer  self-efficacy  was  the  only  predictor  variable  that  was 
statistically  significant  out  of  variables  included  in  the  predictive  model.  The  variables  included  in  the  model  were 
computer  self-efficacy,  academic  self-concept,  age,  gender,  academic  status,  years  of  computer  use,  frequency  of 
computer  use,  computer  training,  Internet  in  a class,  and  participation  in  a workshop  for  Web-based  courses. 

Some  e-learning  experts  present  many  strategies  or  tips  for  the  success  of  e-learning.  Interestingly,  Augusto 
Failde,  senior  vice  president  of  global  development  at  NYUonline,  proposes  1 1 strategies  that  companies  can  use  to 
help  ensure  high  course  completion  rate  (Frankola,  2001a).  These  strategies  are  as  follows:  (1)  develop  a culture  that 
takes  online  learning  just  as  seriously  as  classroom  training,  (2)  do  individual  comparisons,  (3)  hold  managers 
accountable  for  the  success  of  their  employees,  (4)  use  managers  as  role  models,  (5)  create  a social  dimension  to  e- 
learning,  (6)  make  expectations  clear  up  front,  (7)  provide  formal  rewards,  (8)  track  performance,  (9)  get  personal, 
(10)  hold  a team  competition,  and  (11)  launch  a communications  campaign.  He  articulates,  “Gbod  companies  that 
recognize  the  importance  of  human  capital  must  motivate  and  support  employees  as  they  develop  a commitment  to 
life-long  learning’’  (Frankola,  2001).  Broadbent  (2001)  also  gives  e-learning  engagers  some  tips  for  e-learning 
success.  These  tips  include;  (1)  focusing  on  a clear  business  objective,  (2)  don’t  set  very  high  expectations,  (3)  hire 
consultants  or  some  sort  of  service  provider  to  handle  all  of  e-learning  needs,  (4)  don’t  force  e- learning  on  resisters, 
(5)  don’t  evaluate.  Black  (1998)  emphasizes  the  following;  (1)  offer  short  classes,  (2)  make  graphics  simple  and 
easy  to  read,  (3)  foster  collegiality  by  asking  students  to  contribute  information  about  themselves  and  their  interests, 
(4)  vary  the  way  you  interact  with  learners,  (5)  avoid  superfluous  media,  and  (6)  use  a combination  of  synchronous 
and  asynchronous  instruction  to  reinforce  new  material,  design  assignments,  and  improve  learner  retention.  Horton 
(2000)  contends,  “Successful  virtual  classroom  courses  usually  depend  more  on  human  interaction  than  on 
technological  infrastructure’’  (p.  398).  Hence,  he  points  out  that  selecting  a qualified  instructor,  keeping  the  class 
small,  and  responding  promptly  and  reliably  are  important  in  planning  a Web-Base  Training  (WBT)  course.  In 
addition  to  this,  he  suggests  holding  a pre -class  get-together  to  overcome  initial  hurdles;  publishing  a comprehensive 
syllabus;  preparing  learners  to  participate  (e.g.,  the  etiquette  for  online  meetings);  managing  collaborative  activities; 
teaching  the  class-rather  than  just  letting  it  happen  (e.g.,  contact  participants  individually,  help  classmates  get  to 
know  one  another,  stay  on  the  published  schedule,  keep  office  hours,  pace  learners,  do  not  spend  too  much  time 
teaching  the  course  software);  conducting  live  events;  making  participants  visible;  and  staying  in  touch  after  the 
class.  Khan  and  Vega  (1997)  contend  that  the  Web  design  should  be  “logical,  user-friendly,  and  meaningful’’  (p. 

378). 

As  many  researchers  point  out,  motivating  learners  is  a very  important  factor  to  retaining  them  in  e-learning 
courses.  “Successful  WBT  courses  rely  on  the  self-discipline  and  focus  of  motivated  learners’’  (Horton,  2000,  p. 

418).  He  suggests  some  techniques  that  designers  and  instructors  can  use  to  keep  learners  interested,  energized,  and 
enthusiastic.  These  techniques  are:  (1)  set  clear  expectations,  (2)  require  commitment,  (3)  feature  the  WIIFM 
(what’s  in  it  for  me?),  (4)  make  WBT  fun  and  interesting,  (5)  offer  bribes,  (6)  pace  and  prompt  learners,  (7)  provide 
encouraging  feedback,  (8)  build  a learning  community,  (9)  intervene  with  unmotivated  learners,  and  (10)  redeem 
troublemakers. 

Driscoll  (1998)  contends,  “Designing  effective  WBT  requires  knowledge  of  the  unique  characteristics  of 
adult  learners  and  an  understanding  of  the  facilitator’s  role’’  (p.l3).  He  outlines  the  characteristics  of  adult  learners 
as:  real-life  experience,  problem  centered  learning,  continuous  learners,  varied  learning  styles,  responsibilities 
beyond  the  training  situation,  and  meaningful  learning. 
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Like  Tinto’s  model  (1975),  the  two  dimensions  of  integration,  academic  and  social,  form  the  core  of 
Kemper’s  open  learning  model  (1995).  This  model  was  developed  through  the  process  of  validation  of  the  model, 
utilizing  both  quantitative  and  qualitative  data  from  a diversity  of  sources.  This  model  consists  of  several  constructs 
that  affect  outcome  of  students  in  open  learning  courses.  The  construct  of  entry  characteristics  that  influences 
integration  variables  consists  of  demographic  status,  educational  qualifications,  family  status,  and  employment. 
Kemper  (1995)  articulates  that  entry  characteristic  are  not  good  predictors  of  final  outcomes,  because  they  are  just  a 
starting  point  in  determining  how  much  difficulty  a student  is  likely  to  face  in  coping  with  a course.  He  continues, 
“Many  students  with  apparently  adverse  circumstances  do  succeed”  (p.  77).  The  social  integration  construct  consists 
of  enrollment  encouragement,  study  encouragement,  and  family  environment  and  examines  the  degree  to  which 
students  are  able  to  integrate  their  academic  with  the  often  conflicting  employment,  family  and  social  requirements. 
Kemper  (1995)  asserts  that  “social  integration  can  be  achieved,  even  in  the  face  of  an  inhospitable  social 
environment,  if  a time  and  space  for  study  are  negotiated”  (p.  88).  The  external  attribution  construct  consists  of 
insufficient  time,  unexpected  events,  and  distractions.  The  lower  levels  of  social  integration  affect  the  negative 
academic  integration  of  students.  In  the  model,  academic  integration  is  spilt  into  the  positive  (academic  integration) 
and  negative  (academic  incompatibility)  tracks.  Each  construct  consists  of  four  indicators  such  as  study  approach, 
motivation,  course  evaluation,  and  language  ability.  Academic  integration  is  understood  as  “encompassing  all  facets 
of  a course  and  all  elements  of  contact  between  an  institution  and  the  students  whether  these  are  of  an  academic, 
administrative  or  social  nature”  (Kemper,  1995,  p.  99).  In  addition,  GPA  functions  to  some  extent  as  an  intervening 
variable  between  academic  incompatibility  and  dropout.  At  the  final  step  of  the  model,  a cost/benefit  analysis,  the 
student  has  to  make  a decision  about  either  dropping-out  or  completing  study.  This  final  step  includes  a recycling 
loop  that  provides  a mechanism  for  switching  from  one  track  to  the  other. 

Factors  Identified  from  the  Literature  Review  of  Dropout  in  E-learning 

Based  on  variables  (see  Table  1)  identified  from  the  literature  review  of  dropout  in  e-learning  and  the 
models  of  dropout  dealt  with  above  five  constructs  were  categorized:  individual  background,  motivation,  academic 
integration,  social  integration,  and  technological  environment. 


Table  1 Factors  Identified  from 


^^Constmcts  t j-  -j  i 
Individual 

Studie^\  Background 


ASTD  &MASIE 
Center  (2001) 


the  Literature  Review  of  Dropout  in  E-learning 


Augusto  Failde 
(as  cited  in 
Frankola,  2001a) 


Broadbent  (2001) 


Black  (1998) 


Brown  (1996) 


Chyung  (2000, 
2001a,  2001b) 


Chyung  et  al. 
(1998) 


Motivation 


• Incentives 

• Lack  of 
accountability  for 
completing  classes 

• Poorly  designed 
courseware 


• Extrinsic  motivation- 
formal  reward,  team 
competition,  clear 
expectations,  etc. 


• Clear  expectations 

• Need  satisfaction 


• No  evaluation 

• Challengeable 
expectations 

• Clear  business 
objective 

• Short  classes 


> Attraction 

> Confidence 

> Relevance 

> Satisfaction 


> Satisfaction 


Academic 

Integration 


Social 

Integration 
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Driscoll  (1998) 


• Characteristics 
of  adult  learners 


Frankola  (2001b) 


Frontline  Group 

(2001) 

Greer  et  al. 
(1998) 


Gilroy  (2001) 


Florton  (2000) 


Hossein  Arsham 
(as  cited  in 
Elearningpost, 
2001) 


Kemper  (1995) 


Khan  & Vega 
(1997) 

Urn  (2001) 
NYUonline 
(Cited  in 
Frankola,  2001b) 


Osborn  (2001) 


Shepherd  (2001) 


• Individual 
learning 
preferences 

• Learning  styles 


• Entry 
charateristics 


• Educational 
level 

• Number  of  credit 
hours  taken  in  the 
current  semester, 
and  number  of 
previous  distance 
learning  courses. 


• Lack  of  learner 
control  of  content 

• Poorly  designed 
course 

• Poorly  designed 
course 

• Self-motivated 

• Satisfaction 

• Poorly  designed 
course 

• Personalization 

• Live  event 

• Incentive 

• Pre-class  meeting 

• Staying  on  the 
published  schedule 

• Pacing  learners 

• Making  participants 
visible 


• Intrinsic/extrinsic 
motivation 

• Positive/negative 
course  evaluation 


• Poorly  designed 
course 

• Self-efficacy 

• Live  session 


• Computer  confidence 

• Lower  motivation 


• Incentive 

• Learner’s  perception 
of  the  likelihood  of  the 
learner  obtaining  their 
reward  or  avoiding  the 
penalty 

• Inadequate  contents 


• Lack  of 
management/ 
support 

• Inexperienced 
instructor 

• Supporting 
systems 


• Learning 
community 

• Qualified 
instructors 

• Interaction 
individual 
contact 

• Prompt 
feedback 

• Interactivity 
with  student 


• Poor/good 
language  skills 

• Deep/surface 
approach 


• No  support 
of  peer  & 
training 
managers 


• Budgeting  time 

• Supportive 
friends  and  family 


• Family 
environment 

• Enrollment 
encouragement 

• Study 
encouragement 

• Insufficient  time 

• Unexpected 
events 

• Distractions 


• Less-stable 
study 

environments 


• Lack  of  time 


• Technical 
hurdles 


• Technical 
hurdles 
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• Assessment  of 
learner’s  learning 
process  by  trainers 

Sun 

Microsystems 
Inc  (as  cited  in 
Frankola,  2001b) 

• Cmcial 
interactivity; 
faculty  and 
learners 

Towles  et  al 
(1993) 

• Faculty- 

initiated 

interaction 

Vrasidas  and 
Mclsaac  (1999) 

• Prior  experience 
with  CMC 

• The  structure  of  the 
course 

• Interaction 

• Class  size 

• Feedback 

Some  Implications  for  the  Study  of  Dropout  of  Adult  Learners  in  El-learning 

Year  after  year,  e-learning  is  becoming  more  popular  because  it  allows  training  to  be  available  on  demand, 
to  be  delivered  remotely,  and  to  keep  up  with  the  rapid  pace  of  economic  change.  The  flexibility  of  time,  place,  low 
delivery  cost,  and  program  contents  provided  via  e-learning  is  very  appealing  to  workers  who  are  trying  to  improve 
their  careers  related  to  job  performance  or  individual  development  as  well  as  to  training  mangers.  Undoubtedly,  e- 
learning  based  on  the  today’s  advanced  technologies  has  been  considered  as  the  best  learning  delivery  media  for  this 
purpose.  At  this  point,  it  needs  to  understand  the  dropout  of  adult  learners  in  e-learning  for  more  effective  and 
efficient  e-learning  operation. 

The  literature  review  has  at  least  two  significant  implications  for  the  study  of  dropout  of  adult  learners  in  e- 
leaming.  First,  it  needs  to  build  a holistic  model  that  accounts  for  the  phenomena  of  adult  learners’  dropout  in  e- 
learning.  Whatever  the  setting,  it  is  difficult  to  comprehend  the  reason  for  the  learner’s  dropout  in  adult  education 
and  training  programs  because  the  reasons  for  dropout  among  learners  are  numerous  and  complex.  Theory  in  the 
area  of  learner  dropout  supports  a multivariate  framework  to  account  for  the  complexity  inherent  in  analyzing  the 
learner’s  participation  in  multiple  spheres  of  activity  (Osborn,  2001).  Second,  in  addition,  there  is  a need  for 
practical  contributions  of  the  new  model  of  dropout  in  the  field  of  adult  education,  especially,  e-learning.  This 
means  that  any  new  model  based  on  or  including  a variety  of  perspectives  should  have  the  power  to  provide 
practical  contributions  to  the  field  of  adult  education.  For  instance,  if  adult  learners  drop  out  of  a course  due  to 
motivational  factors,  some  prescriptive  strategies  developed  in  terms  of  a motivational  aspect  could  be  provided  for 
adult  education  practitioners  of  e-learning  programs.  Also,  e-learning  program  designers  and  instructors  could  use  it 
to  prevent  or  decrease  the  dropout  rate  in  e-learning  practice. 
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Abstract 

The  purpose  of  this  paper  is  to  present  preliminary  findings  of  a study  dealing  with  the  nature  of  power 
relationships  in  a specific  online  learning  setting,  based  on  critical  discourse  analysis.  The  findings  are  that(  1 ) 
There  were  no  differences  of  power  inequality  in  gender,  race,  and  social  position  among  adult  students  in  terms  of 
discussion  initiatives,  (2)  There  was  much  evidence  that  women  used  more  powerless  words  than  man  did,  and  (3) 
There  were  no  differences  in  the  use  of  powerful  language  between  those  who  have  relatively  high  social  position 
and  those  who  do  not. 

Introduction 

Power  is  a key  element  in  all  human  interactions.  Tisdell  (1993)  points  to  the  structural  inequality  that  exist 
in  society,  the  “power  disparity  between  racial  minorities  and  the  white  majority,  between  the  poor  and  the  wealthy, 
the  undereducated  and  the  educated,  and  women  and  men”  and  how  “these  power  relations  are  reproduced  and 
maintained  through  the  educational  process”  (p.  203). 

Critical  discourse  analysis  can  be  used  identify  and  map  power  relationships  in  educational  settings.  Pratt 
and  Nesbit  (2000)  argue  that  discourses  are  systems  of  thought  based  on  language  in  the  social  sciences;  hence, 
“attention  is  drawn  not  only  to  vocabularies  of  speech  or  writing  but  also  to  how  they  imply  a whole  network  of 
social  relationships  and  regularities”  (p.  118).  In  addition,  they  point  out  that  the  sociocultural  discourse,  which 
posits  that  learning  is  inescapably  bas  ed  on  contextualized  social  relations,  precipitates  questions  about  patterns  of 
social  relations,  power,  and  particularities  of  circumstance  and  settings”  (p.  122).  Cunningham  (2000)  suggests  that 
much  of  the  field  of  adult  education’s  “rhetoric  centers  on  the  learners,  as  if  the  learners  are  disembodied  creatures 
and  as  if  the  social  context,  the  social  structures,  the  social  class  in  which  we  all  exist  do  not  affect  the  process  of 
education”  (p.  573).  Wilson  and  Cervero  (2001),  in  citing  Livingston  (1983),  contend  that  to  practically  confront  the 
world  of  inequity,  we  need  to  understand  the  way  it  is,  have  a vision  for  what  it  should  be,  and  have  strategies  for 
achieving  our  vision.  They  depict  adult  education  as  a site  for  the  struggle  for  knowledge  and  power: 

The  social,  economic,  political,  cultural,  racial,  and  gendered  power  relations  which  structure  all  action  in 
the  world  are  played  out  in  adult  education.  These  systems  of  power  are  an  inescapable  facet  of  social  reality  and 
almost  always  asymmetrical  in  that  they  privilege  some  people  and  disadvantage  others.  Regardless  of  its 
institutional  and  social  location  or  the  ideological  character  of  its  content,  any  policy,  program,  or  practice  of  adult 
education  represents  this  embeddedness  in  a structuring  (but  not  pre -determined)  social  reality.  In  a real  sense,  the 
power  relations  that  structure  our  lives  together  do  not  stop  at  the  doors  of  our  classrooms  or  institutions  that  provide 
adult  education.  (1 6) 

Although  there  is  a body  of  literature  that  discusses  the  types  of  interaction  or  the  factors  influencing 
interaction  in  online  discussions  for  adult  learners,  there  has  been  a lack  of  research  that  specifically  examines  the 
nature  of  power  relations  among  adult  learners  in  online  discussions . The  present  study  employed  critical  discourse 
analysis  in  an  attempt  to  understand  the  nature  of  power  relationships  that  occurred  in  the  written  discourse  of  two 
online  graduate  courses. 

The  purpose  of  this  paper  is  to  present  preliminary  findings  of  a study  dealing  with  the  nature  of  power 
relationships  in  a specific  online  learning  setting.  This  purpose  is  guided  by  the  following  research  questions:  (1) 
What  is  the  nature  of  power  relationships  in  the  online  discussions  of  online  learning  courses?  (2)  In  what  ways  are 
power  and  privilege  expressed? 

Literature  Review 

There  are  a considerable  number  of  studies  describing  online  discussions  or  comparing  online  and  face-to- 
face  discussions  in  adult  and  higher  education.  These  studies  have  mostly  focused  on  such  subjects  as  (1)  how  adult 
students  participate  in  online  discussions,  (2)  the  comparative  advantages  and  disadvantages  of  online  and  face-to- 
face  discussions,  and  (3)  the  relative  merits  of  various  instructional  strategies  for  online  discussions.  However,  most 
of  these  studies  have  been  given  little  attention  to  the  nature  of  power  relationships  and  the  ways  in  which  power 
and  privilege  are  manifested  in  the  online  discussions. 
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Jeris  (2001)  explored  how  time  and  space,  significantly  altered  through  electronic  mediation,  affect  the 
power  relations  among  adult  graduate  students  who  participated  in  an  online  course,  providing  the  comparison  of 
power  relations  within  online  and  face-to-face  classroom  discussion  through  a case  study.  The  author  pointed  out  the 
power  disparity  that  existed  between  women  and  men  participating  in  online  discussions,  illustrating  with  the 
following  example: 

. . .In  relation  to  this  comment,  another  student  remarked,  "I  was  so  embarrassed  by  something  stupid  I said 
during  my  first  MBA  class  that  I made  up  my  mind  right  then,  I was  not  going  to  say  another  word.  If  it  hadn't  been 
for  this  class,  I would  have  kept  that  promise."  Several  students  wanted  to  know  why  this  student  decided  what  she 
said  was  stupid.  She  revealed  that  her  comment  was  pronounced  "utterly  ignorant”  by  a male  classmate  who  was 
also  a professional  colleague  in  a more  senior  position. 

Tisdell  (1993)  examined  how  power  relationships  predominantly  based  on  gender  but  including  race,  class, 
and  age  were  manifested  in  higher  educational  classroom  of  adult  students  through  observations  of  classes  taught  by 
a male  and  a female  professor,  interviews,  and  document  analysis.  She  observed  several  significant  facts  in  terms  of 
power  relations:  (1)  the  students  who  benefited  from  more  interlocking  systems  of  structural  privilege  tended  to 
have  more  power  in  the  classroom  from  the  perspective  of  their  peers  than  the  students  who  had  less  interlocking 
privilege  and  they  played  the  dominant  role  in  the  class,  (2)  the  students  contributed  to  reproducing  structured  power 
relations  in  their  reification  of  patriarchal  values,  (3)  the  male  professor  tended  to  exert  more  control  than  the  female 
professor,  and  (4)  the  middle-aged  women  with  more  education  tend  to  be  more  participatory,  at  least  in  classes 
where  affective  forms  of  knowledge  are  valued. 

Grob,  Meyers,  and  Schuh  (1997)  examined  sex  differences  in  power/powerless  language  such  as 
interruptions,  disclaimers,  hedges,  and  tag  questions  in  the  small  group  context  of  a higher  education  classroom  by 
juxtaposing  two  competing  theoretical  frameworks:  “dual  cultures”  and  “gender  similarities.”  Their  findings 
revealed  that  there  were  no  significant  differences  between  women  and  men  in  their  use  of  interruptions,  hedges,  and 
tag  questions,  which  supports  "gender  similarities”  approach  to  understanding  sex  differences  and  not  the  dominant 
"dual  cultures"  approach  for  investigating  sex  differences.  In  other  word,  there  was  no  evidence  that  men  use  more 
powerful  language  while  women  use  powerless  language. 

McAllister  and  Ting  (2001)  explored  gender  differences  in  computer- mediated  communication  in  web- 
based  college  courses,  analyzing  the  456  discussion  postings  of  34  students  in  2 online  college  courses.  Each 
discussion  posting  was  analyzed  for  seven  variables:  frequency,  length,  readability,  audience,  purpose,  reference, 
and  format.  The  findings  of  the  study  suggested  that  male  and  female  discussion  items  differed  significantly  in 
length,  use  of  indicators  to  specify  a particular  reader,  purpose,  and  use  of  formal  signature.  However,  male  and 
female  discussion  items  did  not  differ  in  frequency,  readability,  intended  audience,  or  references  to  personal 
experience  or  outside  sources. 

Although  all  of  this  literature  contributes  to  our  understanding  of  power  in  online  learning,  it  is  clear  that 
additional  work  is  needed  if  we  are  to  understand  power  dynamics  in  this  rapidly  growing  educational  format.  This 
study  explored  the  ways  in  which  power  and  privilege  are  expressed  in  the  online  discussions  in  higher  education. 

Methodology 

This  study  explored  the  extent  to  which  the  structural  power  inequities  that  exist  in  society  are  reproduced 
in  an  online  classroom  of  adult  graduate  students;  the  study  focuses  primarily  on  power  relationships  based  on 
gender,  but  also  explores  potential  power  inequality  related  to  race,  social  position.  In  this  study,  the  researchers 
used  critical  discourse  analysis  (CDA)  as  a research  methodology. 

CDA  is  “a  type  of  discourse  analytical  research  that  primarily  studies  the  way  social  power  abuse, 
dominance  and  inequality  are  enacted,  reproduced  and  resisted  by  text  and  talk  in  the  social  and  political  context” 
(van  Dijk,  1998, 1 1).  Furthermore,  van  Dijk  asserts  that  effective  research  using  CDA  has  four  key  characteristics: 
(1)  It  focuses  primarily  on  social  problems  and  political  issues,  rather  than  on  current  paradigms  and  fashions,  (2)  it 
is  employs  a multidisciplinary  approach  to  understanding  social  problems,  (3)  rather  than  merely  describe  discourse 
structures,  it  tries  to  explain  them  in  terms  of  properties  of  social  interaction  and  especially  social  structure,  and  (4) 
it  focuses  on  the  ways  discourse  structures  enact,  confirm,  legitimate,  reproduce  or  challenge  relations  of  power  and 
dominance  in  society. 

Fairclough  and  Wodak  (1997,  pp.  271-280)  summarize  the  primary  tenets  of  CDA:  (1)  CDA  addresses 
social  problems,  (2)  Power  relations  are  discursive,  (3)  Discourse  constitutes  society  and  culture,  (4)  Discourse  does 
ideological  work,  (5)  Discourse  is  historical,  (6)  The  link  between  text  and  society  is  mediated,  (7)  Discourse 
analysis  is  interpretative  and  explanatory,  and  (8)  Discourse  is  a form  of  social  action.  CDA  focuses  on  the  role  of 
discursive  activity  in  constituting  and  sustaining  unequal  power  relations  (Fairclough  & Wodak,  1997).  In  a similar 
vein,  van  Dijk  (1996)  articulates  the  elucidation  of  the  relationships  between  discourse  and  social  power  as  one  of 
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the  crucial  tasks  of  CDA.  In  short,  he  maintains  that  CDA  “should  describe  and  explain  how  power  abuse  is  enacted, 
reproduced  or  legitimised  by  the  text  and  talk  of  dominant  groups  or  institutions’’  (p.  84). 

Dellinger  (1995)  says  that  socially  situated  speakers  and  writers  produce  texts  and  the  relations  of 
participants  in  producing  texts  are  not  always  equal;  there  will  be  a range  from  complete  solidarity  to  complete 
inequality.  He  stresses  that  meanings  arise  through  interaction  between  readers  and  receivers,  and  in  most 
interactions,  users  of  language  bring  with  them  different  dispositions  toward  language,  which  are  closely  related  to 
social  positionings.  In  a similar  vein,  Fairclough  (1995)  underscores  that  “analysis  of  texts  should  not  be  artificially 
isolated  from  analysis  of  institutions  and  discoursal  practices  within  which  texts  are  embedded”  (p.  9).  As  Kaplan 
(cited  in  Dellinger,  1995)  expresses,  the  text  is  multi-dimensionally  structured  and  layered  like  a sheet  of  thick 
plywood  consisting  of  many  thin  sheets  lying  at  different  angles  to  each  other. 

Research  Participants  and  Data  Collection 

The  two  online  classes , “G”  and  “O”,  selected  for  this  study  were  Master’s  level  research  classes  at  a large 
state  university  in  the  southeastern  United  States.  The  same  instructor  taught  the  two  online  classes.  In  addition,  the 
contents  and  teaching  strategies  of  the  two  courses  the  instructor  adopted  were  the  totally  same.  The  graduate 
students  enrolled  in  the  two  classes  consisted  of  10  males  and  31  females  between  the  ages  of  twenty-three  and  fifty- 
eight.  All  but  two  of  the  students  were  part-time  students  who  had  full-time  jobs  except  2 persons.  Thirty-one  of  the 
students  were  white  and  the  others  were  of  African  descent.  All  of  the  students  were  adults  (aged  24  or  older), 
though  age  data  were  not  formally  obtained. 

The  online  course  contained  10  units  related  to  the  concepts  of  educational  research.  Each  class  employed  3 
small  group  discussions  in  each  of  the  10  learning  units.  More  specifically,  for  each  learning  unit,  the  instructor  gave 
discussion  questions  or  set  a discussion  task.  Each  member  of  the  group  was  required  to  make  at  least  two 
“substantive  contributions”  to  the  discussion  of  each  unit.  The  instructor  didn’t  participate  in  the  small  group 
discussions,  because  the  intervention  of  the  instructor  could  affect  the  small  group  discussion.  However,  in  the  unit  9 
and  10  discussions  of  total  group,  two  educational  methodology  experts  facilitated  their  discussions. 

Substantive  contributions  were  defined  as  having  three  major  characteristics:  (a)  The  contribution  must 
relates  either  to  the  discussion  task  the  instructor  set  or  to  the  comments  made  by  other  group  members,  (b)  it  must 
be  well  thought-out  and  well  crafted,  and  (c)  it  must  be  at  least  a two  sentences  in  length.  Each  discussion  was  time- 
bound;  there  was  a tightly  controlled  time  period  during  which  students  must  make  their  contributions.  The  total 
span  of  the  discussion  activity  in  the  first  to  eighth  unit  was  1 week  and  that  of  the  remaining  were  2 weeks. 
Ultimately,  there  were  a total  of  1354  postings  made  in  the  two  classes  over  the  span  of  the  semester. 

Data  Analysis 

In  this  study,  discussion  postings  were  analyzed  based  on  techniques  of  critical  discourse  analysis,  closely 
keeping  in  mind  the  primary  tenets  of  CDA.  As  Eairclough  (as  cited  in  Joyce,  2001)  notes  that  “there  is  no  set 
procedure  for  doing  discourse  analysis;  people  approach  it  in  different  ways  according  to  the  specific  nature  of  the 
project,  as  well  as  their  own  views  of  discourse”  (1  17).  Van  Dijk  (1993)  also  points,  “Critical  discourse  analysis  is 
far  from  easy.  ...  it  requires  true  multidisciplinarity,  and  an  account  of  intricate  relationships  between  text,  talk, 
social  cognition,  power,  society  and  culture”  (p.  253).  Joyce  (2001)  stresses,  by  taking  a position,  researchers  must 
be  self-reflexive  in  terms  of  their  interpretations  and  analyses  and  maintain  some  distance  in  order  to  avoid 
producing  analyses  that  map  directly  onto  their  own  personal  beliefs. 

In  this  study,  our  analysis  was  based  on  indicators  of  power/powerlessness  drawn  from  studies  by  Grob, 
Meyers,  and  Schuh  (1997),  McAllister  and  Ting  (2001),  and  Tisdell  (1993)  to  identify  and  analysis  power  relations 
among  participants.  These  indicators  included:  discussion  initiatives,  disclaimers,  hedges,  and  tag  questions.  In  our 
analysis,  we  assumed  the  following: 

a)  A person  who  has  more  discussion  initiatives  is  more  powerful  than  those  who  have  less  discussion 
initiatives. 

b)  The  use  of  disclaimers,  which  are  expressions  of  uncertainty  (e.g.  “I  guess,”  “I  suppose,”  “I  don’t  know 
much  but,”  “I’m  not  an  expert  but”),  indicate  a lack  of  power. 

c)  The  use  of  hedges,  which  include  adverbs  or  adverbial  phrases  that  convey  either  moderation  or  no 
particular  meaning  at  all  (“kind  of,”  “sort  of,”  “probably,”  “perhaps,”  etc.)  indicate  a lack  of  power. 

d)  The  use  of  tag-questions  (i.e.,  short  questions  added  to  the  end  of  a declarative  sentence,  such  as  “doesn’t 
it?”,  “don’t  you  agree?”,  “you  know  what  I mean?”,  etc.)  indicate  a lack  of  power. 

Our  analysis  consisted  of  examining  the  1354  postings — all  of  which  were  printed  and  collected  into  a 
notebook — and  coding  them  for  these  and  other  indicators  of  power/powerlessness. 
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Findings 

Finding  # 1:  There  were  no  differences  of  power  inequality  in  gender,  race,  and  social  position  among  adult  students 
in  terms  of  discussion  initiatives. 

Initial  analysis  shows  that  there  were  no  differences  of  power  inequality  in  race,  gender,  and  social  position 
among  adult  learners  in  terms  of  discussion  initiatives.  Table  land  2 represent  the  number  of  discussion  initiatives 
by  participants  for  each  class.  For  Class  G and  O,  the  discussions  seem  to  be  dominated  by  a few  learners  who  have 
more  discussion  initiatives.  However,  we  need  to  consider  the  context  of  learning  activities.  Specifically,  in  the 
cases  of  NP,  MM,  JH  and  CD  (one  black  woman,  one  black  man,  and  two  white  women)  of  Class  G,  although  they 
have  much  more  discussion  initiatives  than  others,  we  have  to  consider  the  number  of  non-replied  discussion 
inactivates.  This  is  because  the  number  of  non-replied  discussion  initiatives  accounts  for  the  big  part  of  the  total 
number  and  most  non-replied  discussion  initiatives  deviate  from  the  discussion  context.  For  instance,  in  most  cases, 
non-replied  discussion  initiatives  were  posted  at  the  last  minute  because  they  were  part  of  compulsory  discussion 
assignments.  Included  were  such  postings  as  greetings  that  didn’t  need  to  be  replied  by  others.  In  replied  discussion 
initiatives,  as  seen  in  Table  land  Table  2,  VL  and  MM  of  Class  G and  MR  and  DS  of  Class  O have  more  discussion 
initiatives  than  others.  They  are  all  women  who  did  not  have  high  social  positions  except  for  VL  who  has  the 
relatively  high  social  position  of  a vice  president  for  a company. 


Table  1 The  Number  of  Discussion  Initiatives  of  Adult  Learners  for  Class  G 


'\L^arner 

Initiatives^ 

EA 

BB 

CD 

TE 

JF 

LG 

EG 

KG 

JH 

DJ 

RJ 

LJ 

NK 

VL 

BL 

MM 

EO 

AS 

RW 

JW 

Replied 

0 

B 

B 

B 

D 

B 

2 

B 

B 

B 

B 

B 

5 

10 

6 

10 

2 

6 

2 

5 

■ 

B 

Non- 

Replied 

B 

1 

11 

0 

B 

1 

0 

1 

B 

B 

B 

B 

1 

1 

5 

■ 

0 

16 

10 

5 

B 

2 

Sum 

B 

B 

15 

B 

6 

B 

2 

B 

14 

2 

6 

B 

6 

11 

11 

18 

2 

22 

12 

10 

B 

9 

Table  2 The  Number  of  Discussion  Initiatives  of  Adult  Learners  for  Class  O 


'\Learner 

Initiative^ 

SA 

LA 

DC 

SC 

KE 

CH 

KJ 

GJ 

SJ 

PL 

JM 

VN 

TR 

MR 

JR 

ER 

DS 

DY 

Replied 

5 

1 

5 

4 

11 

3 

4 

2 

7 

3 

5 

3 

4 

10 

6 

4 

12 

6 

6 

Non- 

Replied 

6 

6 

1 

3 

3 

3 

1 

6 

3 

4 

4 

1 

4 

1 

1 

1 

1 

6 

5 

Sum 

11 

7 

6 

7 

14 

6 

5 

8 

10 

7 

9 

4 

8 

11 

7 

5 

13 

12 

11 

Eve 

n though  there  were  no  differences  of  power  inequality  in  race,  gender,  and  social  position  among  adult  learners  in 
terms  of  the  discussion  initiative,  there  were  two  types  of  replied  discussion  initiatives. 

The  first  type  is  to  raise  questions  to  induce  the  following  responses.  However,  this  type  was  likely  to  use 
powerless  language  such  as  disclaimers,  hedges,  and  tag  questions.  Here  are  data  to  support  this  finding: 

...I  suppose  the  next  logical  question  is  can  it  be  possible  to  have  research  generated  under  these  ideal 
ethical  conditions?  Any  thoughts?  (RW,  WW,  Unit  1,  GC) 

[Note:  The  first  two  capital  letters  indicate  student’s  name.  The  followings  indicate  race  and  gender  (e.g.,  BW= 
black  woman,  BM=  black  man  WW=white  woman,  WM  = white  man).  Finally,  the  last  letters  points  the  class  (e.g., 
GC  = Q class  and  0C=0  class)] 

...While  I think  that  common  knowledge  and  research  are  equally  important,  I must 
ask...  Can  knowledge  be  produced  and  imparted?  (DS,  WW,  Unit  1,  OC) 

...Maybe  it  just  seems  too  complicated  to  her?  I know  lots  of  people  who  are  working  in  various  areas  of 
adult  ed  who  are  essentially  limiting  themselves  because  they  don 't  or  won 't  use  technology  to  access  information  on 
topics  that  would  help  them  with  their  work.  Does  that  make  sense?  (BB,  WW,  Unit  3,  GC) 

...I  can't  think  of  any  major  decision  in  life  that  should  not  be  researched  first.  Can  anyone  else?  (LG,  WW, 
Unit  3,  GC) 

...Unfortunately,  I can't  think  of  a good  example  of  a variable  that  shouldn’t  be 
studied  right  now  - can  anyone  else?  (SS,  WW,  Unit  4,  OC) 
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...Yet,  I think  it  safe  to  say  that  it  would  be  highly  offensive  and  presumably  unethical  to  research  driving 
habits  based  on  ethnicity,  any  thoughts  on  that?  (DJ,  WM,  Unit  4,  GC) 

...What  if  the  first  100  students  they  meet  are  just  exiting  a class  or  seminar,  therefore  they  could  all  be  the 
same  year  or  major/interest  which  might  affect  the  results?  And  wouldn’t  they  be  excluding  students  who  didn't  have 
a class  or  reason  to  be  at  that  spot  on  campus  that  day?  What  if  the  group  all  happened  to  be  either  frequent  users 
of  the  union,  or  else  non-users?  More  ideas,  anybody?  (MR,  WW,  Unit  5,  OC) 

Wish  I could  post  a graphic  rather  than  explaining  this.  Hope  you  see  what  I trying  to  explain.  Is  this  what 
the  rest  of  you  understand  with  regard  to  Question  7?  (NP,  BW,  Unit  7,  GC) 

...I  think  we  could  add  a lot  more  to  this  question,  it  seemed  hard  for  me  to  answer  it,  so  I wanted  to  try 
and  tackle  it  first.  Anyone  have  any  other  ideas??  Hard  for  me  to  grasp  everything  in  this  chapter,  but  I'm  trying! 
:0)  (JR,  W,  Unit  7,  OC) 

...Also,  page  448  lists  four  main  threats  to  internal  validity:  mortality,  location,  instrumentation,  and 
instrumentation  decay.  Have  you  seen  this  condition  occur  in  your  own  experience?  (JW,  WM,  Unit  9,  GC) 

The  second  type  of  discussion  initiatives  is  revealed  by  student’s  suggestions  of  their  own  opinions.  In  this  type, 
they  did  not  use  powerless  language.  Below  are  data  to  support  this  finding: 

...Is  research  knowledge  better  than  everyday  knowledge?  I’m  not  sure  if  one  is  better  than  the  other 
generally  speaking.  I believe  that  one  is  based  on  the  other.  Research  is  based  on  everyday  knowledge,  behaviors, 
events  etc..  I see  research  as  a tool  to  examine,  discover  and  validate  those  truths  about  everyday  knowledge.  (AS, 
BW,  Unit  1,  GC) 

...however,  my  point  here  is  that  if  the  information  will  be  tainted  then  the  educational  research  is  not 
important  enough  to  justify  the  inconvenience  and  time  loss  it  causes  participants.  (PL,  WW,  Unit  2,  OC) 

...There  are  limitations  but  the  importance  of  descriptive  statistics  in  educations  or 
just  in  our  daily  lives  cannot  be  denied.  (RJ,  WM,  Unit  7,  GC) 

...Usually,  researchers  of  this  type  gather  information  through  naturally  occurring  situations  because  they 
want  to  see  how  and  why  things  happen.  This  is  my  understanding  of  qualitative  research.  (SJ,  WW,  Unit  10,  OC) 
...lam  more  interested  in  qualitative  research  because  it  is  more  "hands  on"  as  the  researcher;  involves 
more  detail  and  description  which  I tend  to  do  more  of  in  my  line  of  work.  One  tends  to  get  a "better  picture"  of  the 
results  1 think  with  qualitative  research.  (DB,  BW,  Unit  10,  OC) 

Finding  # 2:  There  was  much  evidence  that  women  used  more  powerless  words  than  men. 

According  to  Grob,  Meyers,  and  Schuh  (1997),  “typically,  research  on  powerful/powerless  language  use 
and  gender  has  linked  power  language  with  men  and  powerless  language  with  women”  (p.  283).  The  finding  of  this 
study  is  also  consistent  with  this.  While  women  participants  used  relatively  powerless  language,  men  participants 
did  not.  The  followings  are  the  examples  of  the  use  of  language  by  men: 

Thanks  BLfor  agreeing  with  me.  The  bottom  line  is  how  the  knowledge  is  applied.  Whether  it  is  through 
everyday  practical  means  or  by  controlled  experimentation,  the  true  variable  is  application.  ( CD,  BM,  Unit  1,  GC) 

I do  hate  to  straddle  the  fence  on  an  issue,  but  I’m  really  at  a loss  to  describe  either  scientific  research  or 
common  knowledge  as  "better.  " I’ll  keep  thinking....  (DJ,  WM,  Unit  1,  GC) 

...Ai  one  of  the  posts  pointed  out  (I  have  a hard  time  keeping  track  when  I'm  reading  through  them),  the 
answer  in  part  depends  on  how  you  define  terms  such  as  "educational  research"  and  "inconvenience.  ’’  Take  for 
example  the  article  in  today 's  AJC  regarding  the  charter  school  in  Lithonia  ( available  at 

http://www.accessatlanta.coni/ajc/epaper/editions/friday/metro_c3a5f3blela551070058.html).  (TR,  WM,  Unit  2, 
OC) 

...Sometimes  the  "best"  decision  just  has  to  rely  on  an  ounce  of  available  research  and  a ton  of  personal 
expertise.  (DJ,  WM,  Unit  3,  GC) 

Hey  guys,  I think  that  question  1 b.  would  not  be  a cluster  sample,  it  would  be  a two  stage  random.  On  p 
111,  this  is  illustrated  where  a sample  of  clusters  is  randomly  chosen,  then  a sample  of  individuals  from  within  these 
clusters  is  then  randomly  chosen.  Hence  the  two  stages.  (GJ,  WM,  Unit  5,  OC) 

That  being  said,  I do  believe  that  personal  privacy  is  one  consideration  that  may  reasonably  limit  the  use  of 
variables,  but  since  this  relates  to  the  individual,  it  should  not  necessarily  limit  inquiries  that  seek  to  understand 
phenomena  related  to  populations  at  large.  (LJ,  WM,  Unit  5,  GC) 

...so  the  bottom  line  is  that  any  quantitative  study  is  out  of  the  question....  (GJ,  WM,  Unit  10,  OC) 

In  contrast,  women  used  more  powerless  language  such  as  disclaimers,  hedges,  and  tag  questions: 
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We  cannot  assume  everything  researched  is  factual  information.  This  is  another  thing  to  think  about  when 
debating  the  benefits  of  knowledge  gained  through  researchers  and  everyday  research,  isn’t  it?  ...(VL,  WW,  Unit  1, 
GC) 

...From  my  novice’s  perspective,  I think  that  to  obtain  value  from  knowledge  produced  by  research, 
especially  better  value  than  from  everyday  knowledge,  the  research  has  to  have  been  conducted  and  presented  in  an 
ethical  manner.  (RW,  WW,  Unit  1,  GC) 

I think  both  types  of  knowledge  is  not  so  much  better  than  the  other  but  actually  "go  hand  in  hand"; 
dependent  of  the  other.  (MR,  WW,  Unitl,  OC) 

While  pure  academic  research  does  have  a lofty  aspiration  for  seeking  truths,  we  have  to  remember  that  so 
little  (if  anything)  in  this  world  is  value-neutral.  Although  the  basic  facts  revealed  by  research  may  be  unbiased, 
how  they  are  used  most  assuredly  are  not.  ...in  my  own  humble  opinion,  of  course!  :)  (DJ,  WM,  Unit  1,  GC) 

I’m  not  sure  why  this  is  the  only  group  for  this  unit...  (DL,  WW,  Unit  4,  OC) 

Hi  PL,  I’m  not  totally  sure  about  this,  but  I think  the  quantitative  ones  are  a matter  of  degree.  Yes,  I know  a 
person  can  have  blue  eyes  or  VERY  blue  eyes,  but  in  general,  they'd  just  go  in  that  category.  But  you're  right,  there’s 
still  room  for  deciding  how  you  want  to  classify  something,  I guess  it  depends  on  the  research  and  its  purpose.  (MR, 
WW,  Unit  4,  OC) 

...Are  those  part  of  the  population  or  the  ecological  (setting)?  Sorry  to  have  questions  instead  of  answers 
but  I want  to  make  sure  that  I understand  this.  (VN,  WW,  Unit  5,  OC) 

...I  guess  then,  that  for  7,  the  larger  the  standard  deviation,  the  more  heterogeneous  the  scores,  is  true, 
because  they  are  spread  out  more,  or  vary  more.  Is  that  right?  Help!  ) (MR,  WW,  Unit  7,  OC) 

I think  that  rather  than  being  negative  (or  positive)  the  score  would  be  very  low,  close  to  0.0  — meaning  no 
correlation.  I understand  negative  correlation  to  be  as  one  increases,  the  other  decreases  (a  pattern  if  plotted  would 
slant  from  top  left  to  bottom  right)  — in  other  words,  a negative  relationship  (but  a relationship  none-the-less.)  ...Am 
I completely  off  base?  (KJ,  BW,  Unit  7,  OC) 

...lam  not  sure  an  outsider  doing  a personal  interview  would  work  all  that  well,  but  trust  might  be 
established.  (DS,  WW,  Unit  9,  OC) 

...To  me  qualitative  research  seems  like  it  would  be  more  interesting  to  conduct  rather  than  quantitative. 
(SA,  BW,  Unit  10,  OC) 

Finding  # 3:  There  were  no  differences  in  the  use  of  powerful  language  between  those  who  have  relatively  high 
social  position  and  those  who  do  not. 

Some  participants  in  the  two  classes  have  relatively  higher  social  positions  than  others . For  Class  G,  DJ 
(white  man,  director  for  an  university  institution),  NP  (black  woman,  administrator  for  K county  public  schools),  RJ 
(white  man,  chief  operating  officer  of  the  company's  sponsored  M SC  Union),  and  VL  (white  woman,  vice  president 
for  a company)  have  relatively  higher  social  positions  than  others.  For  Class  O,  SC  (white  man,  director  of  field 
operations  and  training  for  an  university  institution),  CH  (white  man,  program  manager  for  an  airline),  TR  (white 
man,  circuit  juvenile  court  judge),  and  DY  (white  woman,  program  director  for  a 4-H  Center's  environmental 
education  program)  have  relatively  high  social  positions . 

Like  the  first  finding  above,  there  were  no  differences  in  the  use  of  powerful  language  between  those  who 
have  relatively  high  social  positions  and  those  who  do  not.  In  fact,  they  often  used  powerless  language  similar  to  the 
other  students . Some  evidence  for  this  finding  are: 

...But  it's  not  valuable  just  because  it's  research-based;  it’s  valuable  because  it’s  relevant.  That  make 
sense?  (TR,  WM,  Unit  I,  OC) 

...I  wonder  if  it  is  ever  common  for  researchers  to  state  a hypothesis  they  are  biased  to  believe  is  false?  In 
other  words,  just  doing  to  research  to  prove  it  incorrect?  (DJ,  WM,  Unit  4,  GC) 

He  or  she  does  not  feel  strongly  about  an  issue  to  respond  directionally,  so  a nondirectional  hypothesis  is 
formed.  By  stating  things  as  "either"  "or",  the  results  will  lead  one  way  or  another.  This  would  "save  face"  when 
the  hypothesis  is  tested  and  conclusions  made.  Just  my  thoughts...  (VL,  WW,  Unit  4,  GC) 

...I  want  to  make  sure  I understand  this  properly,  so  give  me  your  thoughts??????  ...  (NP,  BW,  Unit  7,  GC) 

I am  not  sure  an  outsider  doing  a personal  interview  would  work  all  that  well,  but  trust  might  be 
established.  (DY,  WW,  Unit  9,  OC) 

Am  I being  narrow  minded  or  limited  in  my  thinking?  What  should  a researcher  do  in  the  case  of 
witnessing  “harmful,  illegal,  or  wrongful  behavior?”  (RJ,  WM,  Unit  10,  GC) 

Conclusion 
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Discussions  in  online  learning  settings  are  very  different  from  those  in  the  face-to-face  environment. 
Specifically,  “group  interactions  are  difficult  and  complex  in  an  online  environment  where  a clear  sense  of  personal 
presence  is  difficult  to  maintain’’  (Williams,  Watkins,  Daley,  Courtenay,  Davis,  & Dymock,  2001,  p.  2). 

Accordingly,  we  can  assume  that  power  relations  among  participants  in  an  online  learning  environment  reveal  very 
different  aspects  from  face-to-face  classroom  discussions.  That  is  because  social  cues  such  as  eye  contact,  body 
language,  facial  expression,  and  voice  tones  are  totally  absent  in  the  online  discussion  environment.  More  often  than 
not,  power  relations  among  people  are  likely  to  appear  with  those  cues  in  the  face-to-face  classroom  discussions. 
However,  in  online  discussions,  written  language  alone  is  the  most  important  factor  that  can  uncover  the  power 
relations  among  people. 

The  findings  of  this  study  reveal  that  (1)  there  were  no  differences  of  power  inequality  in  race,  gender,  and 
social  position  among  adult  students  in  terms  of  the  discussion  initiative,  (2)  there  is  much  evidence  that  women 
used  more  powerless  words  than  men,  and  (3)  there  were  no  differences  in  the  use  of  powerful  language  between 
those  who  have  relatively  high  social  positions  and  those  who  do  not.  Interesting  enough,  male  participants  used 
more  powerful  language  than  female  participants  in  online  discussion  learning  settings  did.  This  finding  is  consistent 
with  the  fact  that  powerful/powerless  language  use  and  gender  has  linked  powerful  language  with  men  and 
powerless  language  with  women  (Grob,  Meyers,  & Schuh,  1997). 

At  this  point,  we  may  be  able  to  conclude  that  the  online  discussion  environment  attenuates  the  power  of 
the  privileged  people  established  by  a tacit  consent  in  terms  of  the  last  two  findings,  (2)  and  (3).  However,  before 
coming  to  any  definite  conclusions  we  need  to  conduct  comparative  studies  on  power  relations  among  learners  that 
occur  in  both  online  discussion  and  face-to-face  classroom  discussion  environments. 

As  Wilson  and  Cervero  (2001)  point  out,  the  systems  of  power  that  structure  all  action  in  the  world  are  an 
inescapable  facet  of  social  reality  and  usually  asymmetrical  in  that  they  privilege  some  people  and  disadvantage 
others.  The  burning  problem  is  to  disclose  the  unequal  power  relations  between  people  who  have  the  privilege  and 
those  who  do  not. 
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Abstract 

There  are  contradicting  beliefs  about  student  information  technology  skills.  The  first  is  that  high  school 
students  know  more  than  faculty  about  computers  and  information  technology.  The  second  is  that  incoming 
freshman  do  not  have  the  information  technology  skills  needed  and  faculty  do  not  have  the  time  to  teach  these  skills 
in  addition  to  their  course  content.  This  was  a topic  of  discussion  on  the  Colorado  State  University  campus. 
Questions  included:  Who  is  responsible  for  teaching  the  students  the  skills  they  need?  and  What  skills  need  to  be 
addressed?  A committee,  including  representatives  from  faculty , library,  academic  computing,  and  college  IT 
support  staff,  was  formed  to  investigate  these  issues.  They  designed  a survey  to  support  or  reject  the  hypothesis  that 
a digital  divide  in  IT-based  knowledge  and  experience  did  exist  in  the  freshman  newly  arrived  on  campus.  This 
paper  presents  the  need,  methodology,  and  results  from  this  survey. 


Information  Technology  Literacy 

Institutions  in  higher  education  have  anecdotal  evidence  from  faculty  and  support  centers  that  define  a 
fundamental  “digital  divide’’  in  computer-based  skills  that  students  bring  to  post  secondary  education.  Although 
Edminston  and  McClellend  (2001)  remind  us  that  predictions  were  made  in  the  early  90’s  that  computer  literacy 
courses  would  no  longer  exist  by  the  year  2000  observations  suggest  a remarkable  range  in  students’  knowledge 
about  information  technology  (IT)  concepts  and  software  skills.  In  fact,  in  a recent  study  conducted  by  Hackbarth 
(2000),  elementary  school  students  were  reported  to  have  only  10-60  minutes  of  access  to  information  technology 
each  week.  Due  to  the  lack  of  access  time  at  school,  it  is  presumed  that  K-12  students  are  gaining  their  technology 
literacy  at  home.  The  National  Assessment  of  Educational  Progress  found  that  41%  of  eighth-graders  in  free  and 
reduced  lunch  programs  have  home  Internet  access  compared  to  72%  of  their  financially  better  off  peers.  Sax,  Ceja, 
& Teranishi  (2001)  suggest  that  these  disparities  in  pre -college  use  of  information  technology,  if  not  attended  to, 
may  seriously  compromise  some  students’  ability  to  succeed  to  the  fullest  extent  in  college.  In  such  an  inconsistent 
environment,  instructors  cannot  assume  prior  knowledge  of  even  the  most  basic  of  IT  skills.  Eaculty  are  under 
increasing  pressure  to  incorporate  technology  into  their  teaching  and  learning  activities,  and  develop  technology 
literate  students  upon  graduation.  Unfortunately,  the  inconsistency  in  student  skills  makes  it  difficult  to  design 
effective  technology  enhanced  instruction. 

This  dilemma  is  not  a new  phenomenon  on  campuses,  but  the  search  for  possible  solutions  has  taken  on  an 
increasing  urgency  in  the  past  decade.  The  ability  to  effectively  use  computers  in  the  workplace  is  now  important  in 
almost  every  profession.  The  U.S.  Department  of  Education’s  report  to  the  Nation  on  Technology  and  Education 
(1996)  recognized  that  technology  literacy  “has  become  as  fundamental  to  a person’s  ablity  to  navigate  through 
society  as  traditional  skills  like  reading,  writing,  and  arithmetic.’’  Information  Literacy  is  defined  in  this  report  as 
“computer  skills  and  the  ability  to  use  computers  and  other  technology  to  improve  learning,  productivity,  and 
performance.’’  (p.  1).  The  increasing  use  of  technology  in  higher  education  reflects  this  ubiquity.  If  the  basic 
knowledge  of  information  technology  is  so  central  to  the  education  of  all  citizens,  it  begs  the  question  of  how 
educators  should  address  the  curricular  issues  involved.  In  1999  the  Computer  Science  and  Telecommunications 
Board  (CSTB)  of  the  National  Research  Council  published  their  seminal  report.  Being  Fluent  with  Information 
Technology.  The  book  was  the  culmination  of  a two-year  national  study  by  the  Board’s  committee  on  Information 
Technology  Literacy  (CSTB,  1999).  The  report  addressed  the  centrality  of  information  technology  in  modern  life  in 
the  United  States  and  the  related  implications  for  higher  education.  It  sought  to  answer  the  question  “what  should 
everyone  know  about  information  technology  in  order  to  use  it  more  effectively  now  and  in  the  future?’’  It  made  a 
significant  distinction  between  information  literacy  and  information  fluency.  The  report  argued  that,  since  digital 
technology  is  evolving  at  such  a rapid  rate,  superficial  attempts  to  promote  simply  literacy  by  stressing  the 
acquisition  of  basic  skills  such  as  word  processing  would  be  too  modest  a goal  (CSTB,  1999).  The  acquisition  of 
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information  fluency  implied  a broadening  of  the  learner’s  knowledge  base  to  include  fundamental  IT  concepts  and 
capabilities  that  would  enhance  the  learning  of  knew  digital  skills. 

This  holistic  approach  is  important  in  higher  education  where  a narrow  focus  on  digital  skills  acquisition  is 
often  perceived  as  a simplistic  solution  that  is  dismissed  with  the  epithet  “too  vocational.’’  The  CSTB  report  stresses 
that  the  development  of  IT  concepts  and  capabilities  must  be  linked  with  skills  acquisition  to  transform  literacy  into 
fluency  in  a process  it  labeled  as  “FITness’’.  The  repost  succinctly  outlined  three  significant  components  of 
FITness: 

• contemporary  skills,  the  ability  to  use  today’s  computer  applications 

• foundational  concepts,  the  basic  principles  and  ideas  of  computers,  networks  and  information,  the 
how  and  why,  and 

• intellectual  capabilities,  the  ability  to  apply  information  technology  in  complex  and  sustained 
situations. 

These  are  further  broken  out  into  ten  components  of  fluency  for  each  focus  area. 

While  the  traditional  approach  to  improving  computer  literacy  is  focused  on  skills,  the  FIT  model  is 
valuable  in  its  balanced  approach.  One  can  argue  that  the  enhanced  study  of  information  technology  should  not  be 
pursued  as  something  to  be  grafted  to  the  higher  education  core  curriculum  but  should  rather  be  an  integral  element 
of  it.  This  was  a topic  of  discussion  on  the  Colorado  State  University  campus.  Two  primary  questions  evolved: 

1.  Who  is  responsible  for  teaching  the  students  the  skills  they  need? 

2.  What  do  we  need  to  teach  the  students? 

A committee  was  formed  to  investigate  the  issue.  This  committee  was  facilitated  by  the  Assistant  Director  in  the 
Office  of  Instructional  Services,  members  included  representatives  from  faculty,  library,  academic  computing,  and 
college  IT  support  staff.  Review  the  Website  at  http://www.colosate.edu/webct/computer_literacy_survey/  for  a list 
of  committee  members,  the  survey  questions,  and  the  full  results. 


How  CSU  Addressed  the  Questions 

The  committee  examined  the  information  technology  literacy  issue  from  numerous  angles.  In  agreement 
with  Hirt  et.  al.  (1999)  who  completed  an  assessment  of  computer  skills  on  the  Virginia  Tech  campus,  we 
determined  that  we  needed  to  examine  which  groups  on  campus  were  using  which  types  of  technology  and  how 
skilled  they  are  in  using  that  technology.  Although  in  1999  the  study  was  unable  to  find  studies  related  to 
information  literacy,  we  were  able  to  find  a handful  (Edmiston  & McClelland,  2001;  Hackbarth,  2001;  Olsen,  2000; 
Sax,  Ceja,  & Teranishi.  2001)  which  led  us  to  believe  that  we  needed  to  assess  the  status  specifically  at  CSU.  We 
first  determined  what  support  systems  were  already  in  place.  The  only  centralized  student  support  for  information 
technology  is  the  Help  Desk  which  is  staffed  by  the  Computer  Training  and  Support  Services.  While  this  desk 
provides  answers  to  questions  such  as  how  to  connect  to  the  Internet,  set  up  dial-up  accounts,  set  up  an  eldentity, 
and  forgotten  passwords,  they  do  not  provide  any  formal  training  for  students.  Additional  student  support  is  based 
on  student  technology  fees.  Each  college  sets  its  own  technology  fees  which  need  to  be  approved  by  the  students  in 
that  college.  This  varies  between  colleges  and  impacts  the  levels  of  support  they  provide  for  their  students.  Eor 
example,  the  College  of  Business  requires  all  incoming  freshman  to  attend  a one-week  boot  camp  before  the  start  of 
the  fall  semester.  During  this  time  the  students  receive  instruction  in  the  basic  technology  skills  they  will  need  for 
initial  course  work.  This  includes  setting  up  email  accounts  and  learning  how  to  access  the  college’s  networked 
software  and  drives.  The  College  of  Agriculture  offers  one-credit  elective  courses  that  cover  use  of  software  such  as 
Microsoft  Word  and  Excel.  Other  colleges  do  not  provide  structured  technology  training.  The  existing  training 
programs  were  based  on  a perceived  student  need.  We  determined  that,  if  we  hoped  to  make  effective 
recommendations  for  student  support  in  information  technology  literacy  and  faculty  support  for  enhancing  student 
information  technology  fluency,  we  would  need  to  collect  data  on  students’  current  literacy  levels.  We  examined 
potential  methods  for  collecting  information  from  the  students. 


Methodology 

The  committee  determined  that  a survey  would  be  the  least  invasive  way  to  collect  the  information.  An 
electronic  survey  would  be  the  easiest  and  least  expensive  on  the  front  end  but  if  we  wanted  full  representation, 
including  those  students  who  did  not  possess  strong  technology  skills,  an  electronic  survey  would  certainly  eliminate 
a significant  part  of  the  target  population.  Therefore  we  decided  a paper-based  survey  would  illicit  the  most 
significant  results.  CSU  had  recently  initiated  a one-credit  Ereshmen  Seminar  course  which  is  required  for  all 
freshman  and  transfer  students.  This  would  give  us  access  to  all  freshman  enrolled  in  the  fall  of  2001  without 
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concern  of  anyone  completing  the  survey  twice.  Once  we  had  determined  the  method  and  population,  each  member 
of  the  committee  submitted  a list  of  questions  they  felt  would  be  appropriate  for  the  survey.  The  facilitator  compiled 
a list  of  all  the  questions,  eliminated  duplicates  and  grouped  the  questions  into  categories.  At  this  point,  we  had 
many  varied  questions  and  the  survey  would  take  too  long  to  complete.  The  list  of  questions  was  refined  based  on 
hoe  we  believed  we  could  apply  the  responses.  Some  of  the  questions  would  provide  interesting  information  but 
were  not  applicable  to  the  actual  intent  of  the  survey.  We  narrowed  the  list  down  to  7 1 questions  and  the  response 
format  was  designed  to  allow  us  use  a scantron  form  to  collect  the  information. 

Drs.  Kaminski  and  Seel  met  with  the  Vice  Provost  to  discuss  the  survey  and  obtain  support.  The  survey 
and  intent  was  presented  to  the  Council  of  Deans  who  made  a couple  of  recommendations  that  were  incorporated 
into  the  survey.  The  Vice  Provost  then  sent  a message  to  all  Freshman  Seminar  faculty/instructors  stating  her 
support  for  and  the  importance  of  the  survey.  She  also  requested  they  allow  for  time  in  their  classes  for  student  to 
complete  the  survey.  Her  office  provided  us  with  a list  of  all  freshman  seminar  courses,  the  instructor,  the  days, 
times,  and  locations  of  the  classes. 

Once  the  survey  method  and  questions  were  approved  and  submitted  to  the  IRB  for  exempt  status,  we  field 
tested  the  survey  with  one  freshman  seminar.  The  students  not  only  completed  the  survey  but  were  asked  for 
feedback  on  the  clarity  of  the  questions  and  additional  comments.  We  made  a few  grammatical  modifications  from 
this  field  test.  We  also  ran  the  data  so  we  could  examine  the  output  from  the  scantron  system  and  ensure  we  would 
be  able  to  analyze  it.  Once  this  phase  was  completed  the  process  became  the  mechanics  of  implementing  the  survey 
and  disseminating  the  results.  Enrollment  in  each  freshman  seminar  is  limited  to  19  students.  There  were  214 
seminars  with  a total  enrollment  of  3,898  students.  To  distribute  the  survey,  the  committee  made  4,000  copies  of  the 
survey,  grouped  them  in  stacks  of  20  with  20  scantron  forms  and  a copy  of  the  letter  from  the  Vice  Provost  and 
placed  them  in  an  envelope  which  was  labeled  with  the  course  information.  For  distribution  purposes  we  enlisted 
assistance  from  the  University  Instructional  Technology  Committee  (UITC).  Each  of  the  nine  colleges  has  a 
representative  on  this  committee.  The  UITC  member  contacted  faculty  and  set  up  a time  within  a three-week  period 
to  complete  the  survey.  They  visited  the  classroom,  distributed  the  survey,  collected  it,  and  returned  them  to  the 
Office  of  Instructional  Services.  Once  all  the  surveys  were  returned,  the  scantron  forms  were  compiled,  incomplete 
forms  were  culled  and  the  surveys  were  delivered  to  be  run  through  our  scantron  system. 

The  results  were  returned  in  a text  file  that  contained  the  information  on  each  form.  We  elicited  the  help  of 
a statistics  instructor  for  data  analysis.  He  imported  the  information  into  SPSS  for  analysis  and  tabulated  all  results 
for  overall  freshman  class  response  frequency  as  well  as  frequency  of  response  by  college.  He  then  completed 
additional  analysis  based  on  sets  of  questions  and  skills. 


Findings 

Our  design  worked  well.  We  received  surveys  from  155  course  sections  for  a return  rate  of  72%.  Of  3898 
students,  we  received  2102  correctly -completed  surveys  for  a response  rate  of  54%.  There  were  some  faculty  who 
refused  to  allow  for  time  in  their  class  to  participate  in  the  survey. 

Demographics 

Of  the  responses,  1933  were  freshmen,  118  sophomores,  28  juniors,  and  7 transfer  students.  The  majority 
of  the  respondents  were  18  years  old  (1313),  660  were  19  years,  and  22  under  18  years.  There  were  793  male 
respondents  and  1268  female  respondents.  Although  we  decided  not  to  include  detailed  questions  about  the  students 
high  school  experience,  we  did  feel  that  information  regarding  the  size  of  their  graduating  class  and  if  it  was  a 
Colorado  school  would  provide  us  with  important  demographic  data.  Of  the  respondents,  1587  or  75.5%  indicated 
they  did  graduate  from  a high  school  in  Colorado. 
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Table  1 Graduating  Class  Size 


Number  of  Students 

129  less  than  50  students 

179  51-100  students 

276  101-299  students 

458  201  - 300  students 

1049  more  than  300  students 

We  included  two  questions  regarding  information  technology  presence  in  high  school.  Only  500  students  indicated 
they  had  taken  a programming  course  in  high  school  and  291  indicated  they  had  taken  a Web  development  class  in 
high  school. 

Hardware  Information 

Some  of  the  more  interesting  data  included  information  on  computer  ownership.  Only  7%  or  150  of  the 
students  indicated  they  did  not  own  either  a laptop  of  a desktop  computer.  Of  these,  1286  or  61%  indicated  that 
their  computer  was  less  than  one  year  old. 

Table  2 Hardware  Ownership 


Hardware 

N 

% 

Desktop 

1136 

54.% 

Laptop 

569 

27.% 

Both 

230 

11% 

Printer 

1722 

82% 

Scanner 

445 

21% 

CDROM 

1823 

87% 

RD-recordable 

1083 

52% 

DVD 

989 

47% 

Network  Interface 

1091 

52% 

PDA 

250 

12% 

Email 

As  communications  over  the  Internet  is  one  of  the  most  common  uses  of  information  technology,  we  asked 
the  students  specific  information  regarding  their  use  of  electronic  mail.  The  majority  of  the  students,  98%,  indicated 
that  they  do  have  an  email  account  and  81%  of  them  indicated  they  knew  how  to  attach  a file  when  sending  email. 
While  49%  indicated  they  use  an  email  account  provided  by  their  department  or  the  campus,  41%  indicated  they  use 
a commercial  email  account.  Only  3%  indicated  that  they  did  not  know  that  they  could  receive  a free  email  account 
from  CSU.  80%  of  the  students  indicated  they  check  their  email  on  a daily  basis. 

World  Wide  Web 

The  Web  has  become  another  common  form  of  communication  as  well  as  its  use  for  information  and  fact 
finding.  When  asked  about  their  experience  using  the  Web,  the  majority  of  the  students,  87%  indicated  they  had 
been  using  it  for  2 or  more  years  with  another  9%  for  the  past  year.  When  connecting  to  the  Web,  68%  indicate  they 
use  the  campus  network,  12%  use  DSL  or  a cable  modem,  and  10%  use  a regular  modem  and  phone  line.  An 
interesting  finding  was  that  8%  indicated  they  did  not  know  how  they  connected  to  the  Web. 
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Table  3 Use  of  the  World  Wide  Web 


Use 

N 

% 

Games 

1009 

48% 

Newsgroups 

776 

37% 

Email 

2006 

95% 

Videos 

888 

42% 

Music 

1847 

88% 

Library-  research 

2019 

96% 

Info  on  CSU 

1915 

91% 

Software 

We  asked  the  students  to  indicate  their  proficiency  in  the  use  of  different  software  packages.  We  not  only 
included  the  frequency  of  responded  but  analyzed  the  responses  based  on  three  types  of  software  including  using 
Microsoft  Office  type  software,  Web  and  multimedia  development  type  software,  and  programming  software.  The 
maximum  sum  in  the  Office  and  Development  categories  is  15.  Males  scored  an  average  of  10  and  5 respectively 
while  females  scored  an  average  of  9 and  3 respectively.  The  maximum  score  for  programming  was  9.  Males 
scored  an  average  of  3 while  females  scored  an  average  of  2.  The  difference  in  skills  between  males  and  females 
was  not  significant.  This  information  was  also  broken  out  by  college. 

Table  4 Software  Skills  by  College 


Colie  ee 

Office  Score 

Development  Score 

Proerammine  Score 

Agriculture 

9.8 

4.8 

2.9 

Applied  Human  Sciences 

9.3 

4.2 

2.4 

Business  10.4 

4.8 

2.7 

Engineering 

9.5 

4.8 

2.8 

Liberal  Arts 

9.1 

4.3 

2.1 

Natural  Resources 

9.3 

3.6 

2.1 

Natural  Sciences 

9.6 

4.1 

2.5 

Vet  Med  & Bio  Sci 

9.8 

4.0 

2.2 

Undecided 

9.3 

4.2 

2.3 

Skills 

Under  the  skills  category  we  included  questions  that  intended  to  ascertain  if  the  students  have  basic  skills  in 
setting  up  a computer  and  using  some  of  the  tools  needed  outside  of  a software  package  (information  fluency).  This 
included  the  ability  to  download  and  install  software  where  82%  responded  yes,  the  ability  to  download  and  install 
plug-ins  where  46%  responded  yes,  and  the  ability  to  download  and  read  pdf  files  where  41%  responded  yes. 

We  also  included  some  questions  about  basic  skills  on  information  gathering.  Specifically,  52%  indicated 
they  had  received  training  in  high  school  on  information  gathering.  If  asked  to  write  a research  paper,  58% 
indicated  they  would  first  use  Yahoo  or  Google  for  a search  while  only  23%  indicated  they  would  use  a database  of 
index  of  abstracts  and  citations.  Only  13%  indicated  they  would  go  to  the  library  first.  Surprisingly  when  asked 
what  information  they  could  find  in  a library  catalog,  19%  indicated  books  and  journals,  2%  indicated  citations  to 
journal  articles,  and  71%  indicated  both,  while  7%  indicated  they  don’t  know. 

Support 

We  decided  if  we  hope  to  provided  effective  solutions  to  the  information  technology  fluency  puzzle,  we 
should  determine  the  method  in  which  students  prefer  to  receive  this  information.  In  addition,  we  determined  this 
information  would  inform  us  on  how  much  students  are  using  technology  based  solutions  when  faced  with 
technology  use  challenges.  When  asked  about  how  they  prefer  to  learn  most  of  the  students  indicated  that  they 
would  rather  receive  one-on-one  training  while  online  training  was  not  highly  rated. 
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Table  5 Preferred  Instructional  Style 


Stvle 

N 

% 

One-to-one  help 

1089 

52% 

CD  Computer-based 

186 

9% 

Videotape 

33 

2% 

Online  training 

63 

3% 

Classroom 

Discussion 

584 

28% 

Perhaps  most  important,  when  a society  is  fluent  in  information  technology,  is  that  people  have  a better 
understanding  of  such  issues  as  data  mining,  privacy,  free  speech,  intellectual  property,  and  even  issues  of 
“photographic  truth’’.  There  is  no  one  right  answer  for  resolving  these  issues.  It  requires  a shift  in  thinking  across 
many  areas  of  higher  education.  Increased  awareness  of  the  state  of  the  divide  will  guide  us  on  where  to  concentrate 
our  initial  efforts. 

Events  have  taken  place  at  Colorado  State  Univeristy  that  may  have  effected  our  survey  data.  During  the 
semester  which  we  implemented  the  survey,  the  institution  implemented  an  eldentity  or  a common  username  and 
password  for  access  to  all  electronic  information.  This  required  all  students,  faculty,  and  staff  to  visit  a Website, 
establish  their  campus  wide  username  and  password,  and  indicate  a preferred  email  account.  Students  who  did  not 
already  have  an  email  account  set  up  a campus  account  at  this  time.  The  following  semester,  students  were  required 
to  have  an  eldentity  to  register  for  classes  which  had  a strong  influence  on  accessing  information  online.  This  may 
explain  why  such  a large  percentage  of  students  responding  to  the  survey  indicated  they  did  have  and  use  email.  It 
may  also  have  influenced  the  results  indicating  that  the  most  frequent  use  of  the  Web  was  for  email,  library  research, 
and  obtaining  information  about  CSU.  The  fact  that  music  and  games  followed  in  use  supports  similar  findings  in 
prior  research.  Also  not  surprising  was  the  highest  level  of  technology  literacy  in  software  use  was  knowledge  of 
the  office  products,  with  the  lowest  familiarity  with  programming. 

We  were  surprised  by  the  number  of  students  who  reported  that  they  own  a new  computer.  Frequently  lack 
of  skills  are  attributed  to  lack  of  access  (Hackbarth,  2001,  Poftak,  2002;  Sax,  et.  al.,  2001).  In  agreement  with  prior 
studies  (Hirt,  et.  al),  we  recognized  the  need  to  identify  additional  ways  to  teach  students,  not  only  designing 
programs  to  teach  the  basic  skills,  but  also  to  increase  technology  integration  and  teach  the  importance  of 
information  fluency.  While  the  majority  of  students  have  the  tools  at  hand  to  increase  their  information  technology 
FITness,  they  still  prefer  classroom  and  one-to-one  instruction  for  learning  new  skills.  This  information  is  in 
agreement  with  experiences  at  other  institutions.  An  article  on  freshman  computer  lifestyles  at  five  institutions 
(Olsen,  2000)  indicates  that  the  majority  of  freshman  students  do  not  attend  computer  training  sessions  or  use  help 
services.  This  information  will  be  taken  into  account  as  we  discuss  what  types  of  skills  students  need  to  learn,  and 
how  these  can  be  integrated  into  the  core  curriculum.  Continued  monitoring  of  the  issues  will  provide  current  needs 
and  strategies  and  technology  advances. 

The  overall  materials  cost  of  implementing  the  print-based  survey  was  minimal  at  $980  and  valuable 
information  was  gained.  Incorporation  of  information  literacy  into  the  core  curriculum  will  take  time  and  patience 
as  faculty  become  more  technology  savvy  themselves.  When  discussing  obstacles  to  technology  integration, 

Walbert  (2000)  indicates  that  lack  of  use  is  mainly  due  to  lack  of  access,  lack  of  time,  and  lack  of  experience  on 
what  to  do.  “. . .the  rapidly  spreading  use  of  personal  computers  and  related  telecommunications  networks  explain 
the  accerlerating  pace  at  which  new  technology-based  options  for  teaching  and  learning  emerge.’’  (Moore’s  law,  as 
discussed  by  Steven  W.  Gilbert,  2001,  p 22).  These  new  options  provide  the  tools  to  enhance  information 
technology  fluency  through  the  classroom.  Implementation  of  additional  student  support  and  training  may  ease 
faculty  concern  about  the  time  involved  in  supporting  non-literate  students  and  encourage  them  to  incorporate 
additional  technology  into  their  teaching  and  learning  activities.  Increased  awareness  of  the  actual  levels  of 
technology  literacy  may  help  faculty  in  the  design  of  these  activities.  Gilbert  (2001)  indicates  that  we  need  ‘deep 
learning’  among  faculty,  support  professionals,  and  administrators,  the  process  of  changing  education  is  itself  an 
educational  process.  Faculty  need  a framework,  taxonomy,  and  introduction  to  relevant  models.  They  need  guided 
practice  and  support.  Thanks  to  the  research  done  by  Rickman  and  Grudzinski  (2000)  on  information  technology 
use  in  the  classroom  we  have  input  from  over  2300  students  on  their  perceptions.  They  report  that  the  average  time 
students  thought  IT  should  be  utilized  only  40  percent  of  the  classroom  time.  This  would  coincide  with  instructional 
design  beliefs  that  a mix  of  delivery  modes  provides  the  most  effective  learning  experiences. 
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Each  college  has  looked  at  their  results  and  has  begun  discussions  on  how  to  address  their  specific  needs. 
While  recent  changes  at  the  Provost’s  level  have  hindered  overall  campus  considerations.  The  advisory  board  for 
the  newly  created  Interdisciplinary  Studies  Program  in  Information  Systems  and  Technology  has  taken  the 
information  and  the  idea  of  “FITness”  seriously.  They  are  recommending  repeating  the  survey  with  freshman  over 
the  next  three  years,  and  completing  a follo\wup  survey  of  the  1999  class  the  spring  before  graduation.  We  may 
find  more  institutions  implementing  a similar  program  to  South  Dakota  State  University  (SDSU,  2002).  They  now 
require  students  to  complete  a proficiency  examination  and  an  information  technology  literacy  exam  approximately 
mid-way  through  their  program  of  study.  In  addition,  the  students  participate  in  a department-based  information 
technology  literacy  assessment  based  on  their  particular  program.  Students  must  pass  the  exam  for  graduation.  If 
they  do  not  pass  the  exams  with  at  least  70%,  they  are  required  to  remediate  and  retake  it. 

With  that  said,  additional  research  is  needed  to  help  faculty  and  administrators  determine  the  best  methods 
for  integrating  information  technology  literacy  and  increasing  students’  information  technology  fluency  in  the 
teaching  and  learning  process.  Educators  need  to  not  only  address  the  issue  of  skills,  but  the  need  for  lifelong 
learning  in  a technology-prolific  society.  This  requires  a change  in  administrative  attitude  toward  recognition  of 
faculty  time  invested  in  information  technology.  When  we  repeat  this  study,  we  will  obtain  additional  information 
regarding  the  amount  of  exposure  students  had  to  information  technology  both  in  their  school  and  home 
environments.  We  may  also  focus  more  on  obtaining  information  regarding  the  skills  students  feel  as  though  they 
are  lacking  vs.  the  skills  they  report  to  have. 
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Abstract 

Much  like  Marshall  McLuhan  predicted,  we  live  in  a media-centric  world.  Researchers  have  shown  that 
youth  today  spend  more  time  watching  television  and  movies  than  most  any  other  leisure-time  activity.  Researchers 
have  shown  that  viewers  automatically  learn  to  cope  with  symbolic  presentation  methods  through  repeated  exposure 
to  these  television  and  visual  patterns.  Furthermore,  the  widespread  availability  of  production  techniques  provides 
easy  access  to  capabilities  that  allow  people  to  use  video  media  to  easily  create  their  own  content.  As  a result  of 
reviewing  these  findings,  one  might  be  able  to  surmise  that  perhaps  educators  need  to  look  differently  at  how  to 
motivate  and  educate  today's  media-centric  youth.  Implications  for  teaching  students  in  this  environment  are 
discussed. 

Background 

We  live  in  a media-centric  world.  Researchers  have  shown  that  youth  today  spend  more  time  watching 
television  and  movies  than  most  any  other  leisure-time  activity  (Pearl,  1982).  In  addition,  the  presentation  speed  of 
passages  on  commercial  television  has  increased  significantly  in  the  past  fifty  years  (Stephens,  1996).  Researchers 
have  shown  that  viewers  automatically  learn  to  cope  with  symbolic  presentation  methods  through  repeated  exposure 
to  these  television  and  visual  patterns  (Abelman,  1995;  Bargh,  1988;  Carr,  1982).  Because  of  their  increased 
exposure  to  rapid  sequence  and  presentation  speed  brought  on  by  fast-cuts/montage  found  in  television  programs 
aimed  at  youth,  it  may  also  be  assumed  that  these  individuals  might  be  able  to  comprehend  these  messages  much 
more  easily  and  comprehensibly  than  can  their  adult  counterparts.  One  can  no  longer  assume  that  exposure  to 
television  or  digital  video  is  simply  a passive  viewing  activity.  Advancements  in  technology  are  changing  the  way 
viewers  are  forced  to  look  at  and  interpret  video  media.  Furthermore,  the  widespread  availability  of  production 
techniques  provides  easy  access  to  capabilities  that  allow  people  to  use  video  media  to  easily  create  their  own 
content.  It  has  been  widely  shown  that  these  acquisition  (i.e.,  production)  opportunities  also  increase  exponentially 
one’s  ability  to  comprehend  content  delivered  in  like  form  (Tyner,  1998). 

Corcoran  (1981)  defined  intelligence  as  a skill  in  a particular  medium  and  suggested  that  the  symbolic 
codes  used  in  that  medium  that  serve  communication  purposes  and  are  internalized  by  a receiver  become  an 
authentic  tool  of  thought.  This  idea  may  be  at  the  root  of  how  to  dispel  errant  notions  about  perceived  intellectual 
weaknesses  in  today's  youth.  Current  failures  in  reading  may  be  really  the  result  of  generational  differences  in 
communication  skills  based  on  an  over-dependence  on  the  right  brain  activities  during  early  childhood  and  teenage 
years.  Just  because  students  do  not  know  the  words  to  communicate  their  ideas  does  not  necessarily  mean  that  they 
are  not  having  ANY  intelligent  thoughts.  Perhaps,  they  are  simply  coming  to  these  individuals  in  different  ways.  In 
short,  educators  need  to  look  differently  at  how  to  motivate  and  educate  today's  media -centric  youth.  If  one  follows 
John  Keller's  ARCS  educational  motivational  model  (1983),  it  shouldn't  be  too  hard  to  see  how  making  things 
relevant  and  providing  successful  outcomes  using  new  media  may  be  translated  into  making  positive  connections 
with  today's  media -centric  youth. 

Using  media  as  a cognitive  development  tool  has  incurred  significant  success  in  several  literacy  projects 
recently.  These  projects  utilized  students'  fascination  with  the  technology  of  television  and  video  production  as  a 
hook  to  encourage  them  to  develop  their  non-verbal  (i.e.  oral  and  visual)  story-telling  skills  that  eventually 
translated  into  their  acquiring  increased  text -based  communication  abilities.  These  students  haven't  needed  too  much 
prompting  to  want  to  write  about  their  own  personal  visual  experiences  and/or  story  lines.  Marco  Torres'  work  with 
inner-city  youth  in  east  Los  Angeles,  as  well  other  similar  projects  in  San  Antonio,  Texas,  and  in  Ohio  have  been 
very  successful  in  bringing  otherwise  lost  children  back  into  educational  the  fold  so-to-speak.  By  first  teaching 
students  how  to  communicate  non-verbally  and  then  having  them  utilize  their  own  projects  to  develop  their  verbal 
communication  skills  has  paid  big  dividends. 

Researchers  and  educators  looking  into  the  intrinsic  instructional  value  of  video  media  have  traditionally 
presented  conflicting  views  on  the  role  visual  perception  plays  in  attention,  motivation,  and  recall.  There  have  been 
several  studies  that  have  looked  into  the  potential  affect  mediated  coding  systems  have  on  cognition  (Davis,  Scott, 
Pair,  Hodges,  & Oliverio,  1999;  Nugent,  1982;  Seidman,  1981;  Walma  van  der  Molen  & Van  der  Voort,  2000). 
While  many  theorists  have  looked  into  the  use  of  video,  most  of  them  had  little  good  to  say  about  its  ability  to  bring 
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anything  new  to  the  table  with  regards  to  using  it  as  a communicative  medium  (Berio,  1960;  Calvert  & Scott,  1989; 
Ide,  1974;  Kozma,  1986).  This  may  have  been  due  to  the  limitations  imposed  by  the  technology  in  use  at  the  time. 
Recent  technological  advances  in  commercial  television  production  techniques  allow  today’s  producers  to  readily 
integrate  fast-cuts  and  montage  and  provide  new  tools  to  more  easily  communicate  complex  thought  using  a 
pictorial  narrative  structure.  A correctly  constructed  rapid-cut  montage  passage  has  been  found  to  add  clarity 
because  of  the  phenomenon  that  the  interpretive  whole  of  a montage  segment  literally  communicates  more  than  the 
sum  of  its  parts  (Hitchon,  Druckler,  & Thorson,  1994;  Stephens,  1996).  In  other  words,  it  is  the  composite  whole  of 
all  the  visual  images  in  a passage  or  segment  when  considered  all  at  once  that  gives  it  extended  meaning.  In 
addition,  newer  editing  techniques  have  evolved  that  emphasize  the  perceptual  continuity  of  a rapidly  presented 
image-based  narrative  structure,  rather  than  the  classical  point  of  view  of  editing  that  stressed  the  importance  of 
applying  strict  rules  in  order  to  obtain  slow  and  smooth  transitions  between  successive  shots  (d’ Ydewalle  & 
Vanderbeeken,  1990). 

Need  for  the  study 

Perhaps  it  is  time  to  update  the  thinking  with  regards  to  using  video  as  an  instructional  medium  in  light  of 
new  technological  developments.  A review  of  the  literature  of  the  past  twenty  years  has  already  yielded  some 
studies  into  the  impact  of  rapid  video  editing  in  an  educational  setting  (Stephens,  1996;  Wetzel,  Radtke,  & Stern, 
1994).  However,  more  recent  advances  in  technology  that  have  superceded  those  studies  and  an  increased  usage  of 
rapid -cuts  and  montage  production  techniques  in  commercial  television  programming  have  created  the  need  to 
update  any  earlier  research  that  does  exist.  Lang  has  conducted  more  recent  studies  that  have  looked  into  the 
effectiveness  of  fast-cuts  (Lang,  1994;  Lang,  1996;  Lang  & Basil,  1998;  Lang,  Bolls,  Potter,  & Kawahara,  1999; 
Lang,  Zhou,  Schwartz,  Bolis,  & Potter,  2000),  but  hers  deal  with  television  in  a casual  viewing  environment. 

Description  of  the  study 

Most  of  the  previous  studies  into  the  impact  of  fast-cuts/montage  have  concentrated  on  commercial 
television  viewing  (Bryant,  1991;  Lang,  1998;  Lang,  1999;  Lang,  2000;  Zillman,  1991).  While  there  have  been 
occasional  studies  (Keller,  1976)  into  the  use  of  fast  cuts/montage  in  an  instructional  setting,  they  have  looked  at 
viewers  as  a collective  whole  with  little  regard  for  differences  in  audience  personalities,  capabilities,  cognitive  style, 
or  personal  traits.  Further,  the  subjects  of  all  earlier  studies  in  either  setting  have  been  college-level  students  enrolled 
in  communications  classes  (Keller,  1976;  Lang,  1999;  Lang,  2000;  Reeves  & Nass,  1996),  rather  than  school-aged 
students  viewed  in  their  original  educational  environments. 

The  current  study  was  designed  to  look  at  the  effect  of  these  rapidly -presented  visual  passages  in  an 
educational  setting.  It  is  hoped  that  a new  look  at  rapid  visual  processing  brought  on  by  this  study  might  lay  the 
groundwork  for  educational  media  producers  to  update  their  thinking  about  rapidly -presented  video  montage  by 
providing  a research  basis  that  appears  to  be  lacking  currently.  Specifically,  the  questions  that  were  reviewed  were: 

1)  Is  it  possible  to  remember  content  solely  from  rapidly  paced  visual  montage  that  is  not  supplemented 
with  some  form  of  verbal  narrative? 

2)  Is  there  a difference  between  gist  and  verbatim  memories  for  rapidly  presented  videos? 

3)  How  do  individual  characteristics  such  as  gender  and  learning/cognitive  styles  affect  a student’s  ability 
to  process  fast-cuts/montage  video  presentations  in  a classroom  setting? 

This  study  was  designed  to  determine  if  there  are  differences  in  the  way  visual  montage  messages  that  vary  only  in 
their  presentation  speed  (i.e.,  fast,  medium,  or  slow)  are  perceived  and  immediately  remembered  in  an  educational 
setting.  Recent  reports  found  in  the  literature,  (Brainerd  & Reyna,  1990;  Brainerd  & Gordon,  1994;  Reyna  & 
Kiernan,  1994)  have  suggested  that  researchers  have  been  able  to  successfully  parse  memory  into  verbatim  (i.e., 
precise/literal)  memory  for  specific  details  andgAt  memory  (i.e.,  contextual  or  contextual  remembrances  similar  to 
that  which  is  assessed  in  reading  comprehension  tests).  This  study  looked  at  immediate  memory  as  a whole  and 
separately,  using  these  same  categorizations.  The  educational  setting  was  determined  to  be  several  ninth  grade 
classrooms  that  were  categorized  by  gender  and  also  further  delineated  by  the  subjects’  individual  cognitive/learning 
style.  The  overall  sample  set  for  this  study  came  from  that  population  group.  For  purposes  of  this  study  the 
identification  of  cognitive  style  was  limited  to  the  reflective-impulsive  scale,  as  originally  developed  by  Jerome 
Kagan  (1965;  1966),  and  later  refined  and  re -catalogued  by  Cairns  and  Cammock  (1984).  Identical  videos  that 
varied  only  in  their  speed  of  message  delivery  were  presented  to  three  randomly  assigned  groups  of  students.  The 
procedural  requirements  for  implementing  the  cognitive  style  test  instrument  indicated  that  the  analysis  was  to  be 
performed  on  a smaller  subset  of  the  sample  base  to  see  if  the  variance  in  delivery  speed  also  affects  gist  and 
verbatim  memory,  and  if  there  might  be  any  interaction  with  their  prospective  cognitive  styles. 
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The  current  study  integrated  a commercially -made  video  montage  (American  Time  Capsule  by  Chuck 
Braverman),  for  which  permission  was  granted  by  the  current  copyright  holder.  Pyramid  Media,  to  use  in  this  study 
(R.  Wright,  personal  communication,  March  21,  2001). 

This  study  had  four  overall  purposes.  The  first  was  to  determine  whether  differences  in  presentation  speed 
of  a video  montage  would  have  an  effect  on  a memory.  Second,  this  study  attempted  to  determine  whether  the 
presentation  speed  affect  males  and  females  differently.  A two-way  analysis  of  variance  was  used  in  these  two 
portions  of  the  study.  Third,  the  study  attempted  to  determine  whether  there  would  be  differences  between 
immediate  verbatim  and  gist  memory  for  the  content  of  the  video  caused  by  changes  in  presentation  speed.  The 
design  of  this  portion  of  the  study  was  a multivariate  analysis  of  variance.  Last,  an  investigation  was  made  to 
determine  whether  the  immediate  memories  for  subjects  classified  as  either  impulsive  or  reflective  would  be 
influenced  by  changes  in  presentation  speed.  The  design  of  this  portion  of  the  study  was  a 2 x 3 factorial  design 
ANOVA. 

Results 

The  data  were  analyzed  with  analyses  of  variance  and  a multivariate  ANOVA.  The  relevant  statistics  are 
presented  in  ANOVA  summary  tables.  In  addition,  descriptive  statistics  are  included  for  means  and  standard 
deviations. 

Hypothesis  1.  There  are  no  significant  differences  in  overall  test  scores  (in  which  verbatim  and  gist  scores 
are  pooled  together),  for  the  overall  sample  set  based  on  presentation  speed  of  the  video. 

The  analysis  resulted  in  anF  (2,  201)  of  .317  (see  Table  4).  An  F ratio  of  .317  is  not  significant  at  (p  <.05), 
therefore,  this  null  hypothesis  was  not  rejected. 

Table  1 Source  Table  of  Analysis  of  Score  Variance  by  Presentation  Speed 


Source 

SS  (Type  III) 

df 

MS 

E 

Sig. 

Between  Groups 
SPEED 

6.377 

2 

3.189 

.317 

.729 

Error 

2022.329 

201 

10.061 

Total 

2028.706 

203 

The  means  and  standard  deviations  for  presentation  speed  are  presented  in  Table  2.  In  Table  2,  ‘Fast’ 
represents  the  results  for  those  who  watched  the  video  presented  at  its  original  speed  (averaging  300-400 
milliseconds  per  picture),  ‘Medium’  at  1/3  of  the  original  (approximately  500-700  milliseconds),  and  ‘Slow’  (at 
approximately  1/2  the  original  speed  (each  picture  was  presented  at  approximately  1 second  each). 


Table  2 Score  Means  and  Standard  Deviation  for  Presentation  Speed 


SPEED 

Mean 

SD 

N 

Fast 

12.99 

3.18 

75 

Medium 

12.70 

3.05 

69 

Slow 

12.57 

3.30 

60 

Total 

12.76 

3.16 

204 

Hypothesis  2.  There  are  no  significant  differences  in  overall  test  scores  between  male  and  female  subjects 
in  the  overall  sample  set. 

The  analysis  resulted  in  abetween  subjects  effectF  (1,  202)  of  1.542  (see  Table  3.  Because  anFratio  of 
1.566  for  this  main  effect  is  not  significant  at  (p  <.05),  this  null  hypothesis  could  not  be  rejected.  The  means  and 
standard  deviations  are  presented  in  Table  4.  The  mean  scores  and  standard  deviations  are  shown  by  gender  (i.e., 
males  and  females).  The  table  shows  a difference  in  scores  and  standard  deviations  between  males  and  females,  with 
males  obtaining  a higher  average  score.  However,  as  the  differences  are  not  significant,  they  are  considered 
statistically  as  random  variations. 
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Table  3 Source  Table  of  Analysis  of  Variance  for  Gender 


Source 

SS  (Type  III) 

df 

MS 

E 

Sig. 

Between  Groups 

GENDER 

15.604 

1 

15.604 

1.566 

.212 

Error 

2013.101 

202 

9.966 

Totai 

2028.706 

203 

Table  4 Score  Means  and  Standard  Deviation  for  Gender 


GENDER 

Mean 

N 

SD 

Male 

13.03 

107 

3.34 

Eemale 

12.47 

97 

2.94 

Total 

12.76 

204 

3.16 

Hypothesis  3.  There  is  no  significant  interaction  between  presentation  speed  and  gender  for  overaii  test 
scores. 


Table  5 Source  Table  of  Analysis  of  Variance  of  Speed  versus  Gender 


Source 

SS  (Type  III) 

df 

MS 

E 

Sig. 

Between  Groups 

SPEED 

8.437 

2 

4.219 

.419 

.659 

GENDER 

15.540 

1 

15.540 

1.542 

.216 

SPEED  * GENDER 

8.766 

2 

4.383 

.435 

.648 

Error 

1995.428 

198 

10.078 

Total 

35268.000 

204 

Table  5 shows  the  interaction  effect  for  between  subjects  of  SPEED  * 

GENDER  resulted  in  an  F ratio  of 

.435.  This  F ratio  is  not  significant  at  (p<.05). 

Therefore,  this  null  hypothesis  was  not  rejected.  Table  6 shows  the 

means  and  standard  deviations  comparing  presentation  speed  with  gender.  Males  had  higher  memory  scores  than 
females  but,  these  differences  are  shown  to  be  not  significant.  As  such,  they  are  considered  random  fluctuations. 

Table  6 Score  Means  and  Standard  Deviation  for  Speed  versus 

Gender 

SPEED 

GENDER 

Mean 

SD 

N 

Fast 

Male 

13.56 

3.10 

34 

Female 

12.51 

3.20 

41 

Total 

12.99 

3.18 

75 

Medium 

Male 

12.97 

3.46 

38 

Female 

12.35 

2.48 

31 

Total 

12.70 

3.05 

69 

Slow 

Male 

12.57 

3.46 

35 

Female 

12.56 

3.12 

25 

Total 

12.57 

3.30 

60 

Total 

Male 

13.03 

3.34 

107 

Female 

12.47 

2.94 

97 

Total 

12.76 

3.16 

204 

Hypothesis  4.  There 

are  no  significant  differences  in  verbatim  test  scores,  for  the  overall  sample  set,  based 

on  presentation  speed  of  the  video. 

The  analysis  of  verbatim  scores  resulted  in  anF  (2,  201)  of  1.082  (see  Table  7). 
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Table  7 Source  Table  of  Analysis  of  Variance  for  Verbatim  Scores  for  Speed 


SS  (Type  III) 

df 

MS 

F 

Sig. 

Between  Groups 

12.035 

2 

6.018 

1.082 

.341 

Error 

1117.592 

201 

5.560 

Total 

1129.627 

203 

Because  the  F ratio  for  verbatim  recall  was  not  significant  at  (p<.05),  this  null  hypothesis  could  not  be 
rejected.  The  means  and  standard  deviation  are  presented  in  Table  1 1. 

Hypothesis  5.  There  are  no  significant  differences  in  gist  test  scores  for  the  overall  sample  set,  based  on 
presentation  speed  of  the  video. 

The  analysis  of  gist  scores  resulted  in  anF  (2,201)  of  5.491  (see  Table  8).  Because  the  F ratio  for  verbatim 
recall  was  significant  at  (p<.05),  this  null  hypothesis  was  rejected. 

Table  8 Source  Table  of  Analysis  of  Variance  for  Gist  Scores  for  Speed 


Source 

SS  (Type  III) 

df 

MS 

E 

Sig. 

Between  Groups 

32.462 

2 

16.231 

5.491 

.005* 

Error 

594.127 

201 

2.956 

Total 

626.588 

203 

* Significant  at  (p<.05) 


Because  the  results  were  found  to  be  significant,  and  because  there  were  more  than  two  groups  to  be 
compared,  a Bonferroni  test  was  run  to  compare  individual  pairings  of  gist  scores  to  each  of  the  individual 
presentation  speeds.  Table  9 shows  that  the  pair-wise  comparisons  are  significant  for  gist  scores  when  comparing 
‘Slow’  and  ‘Fast’  speeds,  but  not  significant  between  ‘Medium’  and  ‘Slow’  or  ‘Medium’  and  ‘Fast’. 

In  order  to  further  compare  verbatim  and  gist  scores,  a means  and  standard  deviation  table  (Table  10)  is 
presented.  Table  10  shows  a mean  score  for  fast  speeds  of  6.01  out  of  10  possible  gist  responses  for  those  viewing 
the  video  at  the  fast  speed  and  5.05  (out  of  10)  mean  score  for  those  viewing  it  at  the  slow  speed.  Table  10  shows 
that,  as  a percentage,  those  subjects  who  watched  the  video  at  the  two  faster  speeds  also  tended  to  get  more  gist 
questions  correct  than  those  watching  at  the  slow  speed.  For  the  verbatim  questions,  there  were  no  significant 
differences  based  on  changes  in  speed. 

Table  9 Multiple  Comparisons  Between  Speed  and  Gist  Scores 


SPEED 

SPEED 

Mean 

Difference 

Std. 

Error 

Sig. 

95% 

Confidence 
Interv  al 
Lower  Bound 

Upper  Bound 

East 

Medium 

.25 

.287 

1.000 

-.45 

.94 

Slow 

.96* 

.298 

.004 

.24 

1.68 

Medium 

East 

-.25 

.287 

1.000 

-.94 

.45 

Slow 

.72 

.303 

.057 

-.01 

1.45 

Slow 

East 

-.96* 

.298 

.004 

-1.68 

-.24 

Medium 

-.72 

.303 

.057 

-1.45 

.01 

* Significant  at  (p<.05) 
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Table  10  Score  Means  and  Standard  Deviation  for  Gist  versus  Verbatim  Test  Items 


SPEED 

Mean 

Std.  Deviation 

N 

VERBATIM 

Fast 

6.93 

2.47 

75 

(15  questions) 

Medium 

6.97 

2.26 

69 

Slow 

7.48 

2.33 

60 

Total 

7.11 

2.36 

204 

GIST 

Fast 

6.01 

1.79 

75 

(10  questions) 

Medium 

5.77 

1.64 

69 

Slow 

5.05 

1.72 

60 

Total 

5.65 

1.76 

204 

Hypothesis  6.  There  are  no  significant  differences  in  overaii  test  scores  for  those  subjects  from  the  overaii 
sampie  set  who  are  determined  to  possess  impuisive  or  reflective  tendencies. 

As  seen  in  Tabies  i i and  i2,  the  sampie  size  is  smaiier  (n=i29  versus  n=204)  due  to  the  procedures  invoived  in 
determining  impuisive  and  reflective  tendencies.  A portion  of  the  sampie  was  systematicaiiy  exciuded  due  to  these 
subjects  being  cast  as  either  fast-accurate  or  siow-inaccurate,  which  piaced  them  outside  the  parameters  set  forth  by 
the  administrative  instructions  that  accompanied  the  MFFT-20.  An  anaiysis  of  variance  was  performed  and  obtained 
an  F (2,i23)  of  6.560  for  the  main  effect  for  cognitive  styie.  The  F ratio  for  between  subjects  was  significant  at 
(p<.05),  therefore,  the  nuii  hypothesis  was  rejected.  The  means  and  standard  deviations  are  presented  in  Tabie  12. 
This  table  shows  that  impulsive  subjects  had  significantly  lower  correct  scores  than  reflective  subjects,  regardless  of 
presentation  speed. 

Table  1 1 Source  Table  of  Analysis  of  Score  Variance  by  Presentation  Speed  and  Cognitive  Style 


Source 

SS  (Type  III) 

df 

MS 

E 

Sig. 

SPEED 

2.968 

2 

1.484 

.160 

.852 

STYLE 

60.870 

1 

60.870 

6.560 

.012* 

SPEED  * STYLE 

7.380 

2 

3.690 

.398 

.673 

Error 

1141.340 

123 

9.279 

Total 

21733.000 

129 

* Significant  at  (p<.05) 


Table  12  Means  and  Standard  Deviation  for  Overall  Scores  for  Style 


SPEED 

STYLE 

Mean 

Std.  Deviation 

N 

East 

Impulsive 

11.50 

2.09 

22 

Reflective 

13.50 

3.31 

24 

Total 

12.54 

2.94 

46 

Medium 

Impulsive 

12.36 

3.86 

25 

Reflective 

13.24 

2.14 

21 

Total 

12.76 

3.19 

46 

Slow 

Impulsive 

11.82 

3.30 

17 

Reflective 

13.10 

3.02 

20 

Total 

12.51 

3.18 

37 

Total 

Impulsive 

11.92 

3.17 

64 

Reflective 

13.29 

2.85 

65 

Total 

12.61 

3.08 

129 
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Hypothesis  7.  There  are  no  significant  differences  in  verbatim  test  scores  for  those  subjects  who  are 
determined  to  possess  impuisive  or  reflective  tendencies. 


In  order  to  further  investigate  the  differences  in  scores  obtained  for  impuisive  or  reflective  styies,  a one-way  anaiysis 
variance  was  deveioped  (Tabie  i3).  An  anaiysis  was  performed  and  obtained  an  F (i,  i27)of  2.925  for  verbatim 
scores.  The  F ratio  for  between  subjects  was  not  significant  at  (p<.05),  therefore,  the  nuii  hypothesis  was  not 
rejected. 


Table  13  Source  Table  of  Analysis  of  Score  Variance  for  Verbatim  and  Gist  and  Cognitive  Style 


SS  (Type  III) 

df 

MS 

F 

Sig. 

GIST 

Between  Groups 

12.089 

I 

12.089 

4.410 

.038* 

Within  Groups 

348.144 

127 

2.741 

Totai 

360.233 

128 

VERBATIM 

Between  Groups 

14.267 

I 

14.267 

2.925 

.090 

Within  Groups 

619.423 

127 

4.877 

Totai 

633.690 

128 

* Significant  at  (p<.05) 


Hypothesis  8.  There  are  no  significant  differences  in  gist  test  scores  for  those  subjects  who  are  determined 
to  possess  impuisive  or  reflective  tendencies. 

The  same  anaiysis  of  variance  used  for  Hypothesis  7 was  used  to  show  both  verbatim  and  gist  scores  (Tabie 
i3).  The  anaiysis  obtained  anF  (i,  i27)  of  4.4i0  for  verbatim  scores.  The  F ratio  for  between  subjects  was 
significant  at  (p<.05),  therefore,  the  nuii  hypothesis  was  rejected. 

A means  and  standard  deviation  tabie  (Tabie  i4)  was  deveioped  to  further  investigate  these  differences. 
Tabie  i4  shows  that  the  significant  differences  in  test  scores  found  in  Hypothesis  6 were  derived  from  the  gist 
portion  of  the  memory  test,  which  is  consistent  with  previous  findings  regarding  the  significance  of  gist  versus 
verbatim  memory  from  the  overaii  combined  test  scores. 

Table  14  Means  and  Standard  Deviation  for  Gist  and  Verbatim  Scores  for  Style 


STYLE 

Mean 

Std.  Deviation 

N 

GIST 

Impulsive 

5.20 

i.77 

64 

Reflective 

5.82 

i.54 

65 

Totai 

5.5i 

i.68 

i29 

VERBATIM 

Impuisive 

6.77 

2.i4 

64 

Reflective 

7.43 

2.27 

65 

Totai 

7.i0 

2.23 

i29 

Hypothesis  9.  There  is  no  significant  interaction  between  cognitive  styie  and  presentation  speed  of  the 
video. 

The  interaction  effect  for  between  subjects  of  cognitive  styie  (STYLE)  and  presentation  speed  (SPEED) 
resuited  in  anF  (2,  i23)  of  .398  (see  Tabie  i i).  This  F ratio  is  not  significant  at  (p<.05).  Therefore,  this  nuii 
hypothesis  was  not  rejected. 


Summary 

The  resuits  of  this  study  indicate  that  students  tend  to  remember  gist  information  from  rapidiy  presented 
videos  better  than  those  presented  at  siower  speeds.  These  resuits  appear  to  contradict  eariier  research  that  indicated 
that  the  viewers  wouid  be  abie  to  remember  more  information  from  pictures  if  they  are  presented  at  siower  speeds. 
One  of  the  reasons  might  have  been  because  the  eariier  studies  measured  memory  on  a combined  basis, 
concentrating  soieiy  on  measuring  immediate  verbatim  memory.  The  parsing  of  gist  and  verbatim  scores  in  the 
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current  study  has  uncovered  a potential  new  approach  to  investigate  differences  in  pictorial  cognition.  The  fact  that 
the  significant  differences  in  gist  memory  was  masked  when  the  verbatim  and  gist  memory  scores  were  considered 
together  reveals  that  researchers  may  need  to  identify  new  paradigms  that  take  into  consideration  the  goals  and 
intended  outcomes  of  the  instructional  activities  they  are  investigating.  Not  all  classroom  experiences  need  to  have 
rote  memory  as  a sole  learning  outcome.  Stimulating  gist  memories,  like  teaching  reading  comprehension,  may  have 
its  own  place  in  overall  instructional  schemes. 

To  a degree,  the  results  of  the  cognitive  style  instrument  used  in  this  study  to  categorize  subjects  indicate 
that  some  things  about  learning  styles  have  changed  since  the  original  instrument  was  analyzed  and  developed. 

When  one  compares  the  results  of  MFFT-20  cognitive  style  in  the  current  study  to  the  norms  provided  by  earlier 
research  by  Cairns  and  Cammock  in  1984,  not  only  has  the  median  total  number  of  errors  decreased  (from  28-30  in 
the  Cairns  and  Commock  studies  to  eight  in  the  current  study),  but  also  so  has  the  median  latency  to  first  response 
(from  18  to  9.12).  These  reductions  seem  to  indicate  that  latencies  to  first  response  for  visual  activities  are  growing 
shorter,  and  that  the  quicker  responses  do  not  always  translate  to  higher  error  ratios.  Students  appear  to  be 
developing  a propensity  for  remembering  things  from  rapid  visual  presentations. 

Another  change  that  took  place  is  the  shrinking  of  the  differences  in  visual  cognition  between  males  and 
females.  With  Cairns  and  Cammock  (1984),  responses  were  considered  to  lie  so  dramatically  outside  of  the  norms 
that  they  were  systematically  eliminated  from  their  studies.  In  the  current  study,  any  differences  between  males  and 
females  that  did  exist  were  found  to  be  non  significant.  While  females  still  may  be  found  to  be  more  reflective  than 
their  male  counter-parts,  these  differences  may  be  growing  smaller. 

The  results  of  this  study  als  o indicate  that  using  the  impulsive-reflective  cognitive  style  instrumentation 
may  be  still  a valid  measuring  tool.  While  both  verbatim  and  gist  memories  were  both  negatively  affected  by 
cognitive  style,  it  was  only  gist  memory  for  the  rapidly  presented  videos  that  was  affected  significantly.  When 
considering  the  entire  sample  without  regard  to  style,  the  analysis  of  variance  did  not  yield  any  significant 
differences.  The  cognitive  style  portion  of  this  study  uncovered  some  discrepancies  (i.e.  learning  difficulties)  that 
might  have  otherwise  gone  unnoticed. 

Whether  the  changes  found  here  are  the  result  of  differences  in  casual  television  viewing  habits  or  computer  usage 
(or  both  in  combination)  was  outside  the  purview  of  the  current  study.  However,  many  of  the  results  indicate  that 
something  is  different  about  the  way  today’s  youthful  learners  perceive  visual  inputs,  creating  several  interesting 
scenarios  for  future  studies. 

It  is  noteworthy  that  the  total  correct  number  of  responses  for  all  groups  was  quite  low  (around  50%).  The 
relatively  low  numbers  of  correct  to  total  possible  responses  serves  to  reinforce  that  the  purpose  for  integrating  video 
presentations  into  teaching  and  learning  situations  has  not  changed.  While  a change  in  presentation  style  may 
provide  an  essential  pre-condition  for  increasing  knowledge,  it  still  needs  to  be  coupled  with  sound  instructional 
strategies  for  any  learning  to  take  place. 


Revisiting  McLuhan 

New  media  tends  to  attract  right-brain  cognition,  whereas  text  -based  cognition  is  left-brain.  An  over- 
dependence on  one  side  or  the  other  appears  to  prevent  full  development  of  one's  potential  and  can  leave  a child  ill 
equipped  to  fully  function  in  the  world.  Researchers  (Doman,  1984;  Shichida  , 1994)  have  presented  considerable 
evidence  that  children  are  born  with  right-brained  proclivities.  Some  of  them  never  quite  grow  out  of  that  stage.  The 
cause  for  this  delay  might  be  due  to  outside  influences,  like  extended  viewing  of  television.  Some  of  this  might  also 
be  due  to  misguided  attempts  to  force-feed  text -based  activities.  Doman,  Jr.  (1984)  very  often  preaches  that  one 
should  teach  to  one's  strengths  and  remediate  any  weaknesses.  If  it  is  true  that  many  of  today’s  youth  are  right-brain 
dominated,  then  perhaps  using  right  brained  activities  is  a way  to  help  them  develop  their  left  brain  capabilities  (i.e. 
text -based  or  left-brain  cognition).  In  other  words,  there  might  be  considerable  success  in  using  visual  skills  to  get  at 
and  develop  textual.  Whether  the  changes  found  here  are  the  result  of  differences  in  casual  television  viewing  habits 
or  computer  usage  (or  both  in  combination)  was  outside  the  purview  of  the  current  study.  However,  many  of  the 
results  indicate  that  something  is  different  about  the  way  today’s  youthful  learners  perceive  visual  inputs,  creating 
several  interesting  scenarios  for  future  studies. 

The  findings  of  this  study  also  appear  to  re -confirm  McLuhan’s  concept  of  the  medium  is  the  message 
paradigm.  The  study  Perhaps  it  is  true  that  the  media  people  utilize  most  often  not  only  help  to  define  the  messages 
they  receive,  but  they  also  help  to  re -define  the  people  who  use  that  media. 
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Abstract 

While  a majority  of  the  universities  are  offering  traditional  face-to-face  formats  to  deliver  courses  in 
Instructional  Design,  there  are  also  several  institutions  offering  them  in  on-line  formats.  What  concerns  or 
suggestions  do  international  students  have  when  they  are  taking  an  Instructional  Design  course  as  an  on-line 
course?  This  paper  focuses  on  international  students'  perceptions  and  attitudes  toward  their  first  on-line  course 
and  shares  their  reflections  on  their  first  on-line  learning  experience  in  the  United  States. 

Background 

Numerous  studies  have  compared  the  performance  of  distance  learners  to  that  of  traditional  learners 
(Russell,  1998;  Moore  & Thompson,  1990).  The  broad  consensus  among  these  performance  studies  is  that  there 
appears  to  be  “no  significant  difference”  in  learner  achievement  between  the  two  modes.  Generally  speaking,  the 
attitudes  of  students  were  very  positive  and  supportive  toward  the  on-line  instmction  (Chang,  2000).  However,  there 
are  few  studies  on  international  students’  perceptions  and  attitudes  towards  the  on-line  learning  environment. 

Heikinhimo  and  Shute  (1986)  reported  that  most  Asian  students  who  study  in  the  United  States  had 
problems  with  understanding  lectures,  taking  notes,  answering  questions,  and  writing  essays  as  a result  of  language 
obstacles.  Most  international  students  have  different  learning  styles  and  cultural  backgrounds  compared  to  their 
American  peers.  These  students  are  more  comfortable  with  lecture  based  learning  since  the  concept  of  on-line 
learning  is  still  new  in  their  countries. 

Instructional  Design  is  a required  course  for  all  the  graduate  students  majoring  in  the  field  of  Educational 
Technology.  In  most  universities,  instructional  design  is  offered  in  a face-to-face  teaching  format.  For  the  first  time, 
the  Instructional  Design  course  in  this  study  was  offered  in  an  on-line  format.  There  were  a few  international 
students  who  enrolled  in  this  course  as  their  first  on-line  learning  experience. 

This  study  investigated  international  students’  attitudes  toward  their  first  on-line  learning  experience  to  find 
out  whether  they  would  feel  more  comfortable  and/or  learn  more  effectively  in  the  on-line  environment.  The 
primaiy  research  questions  for  the  study  were: 

1.  What  were  international  students’  attitudes  toward  their  first  on-line  learning  experiences? 

2.  Did  international  students  prefer  the  on-line  learning  environment  to  the  traditional  classroom  learning 

environment? 

3.  Did  the  on-line  course  provide  more  or  less  interaction  than  traditional  courses  for  international  students? 

Method 

Subjects 

The  subjects  were  five  international  students  who  were  studying  at  a university  in  the  Western  area  of  the 
United  States.  Among  these  five  students,  two  students  were  enrolled  in  the  doctoral  programs  (Educational 
Technology  and  Educational  Leadership)  and  three  were  enrolled  in  the  masters  program  in  Educational 
Technology.  The  nationality  of  the  subjects  is  Chinese  - two  female  students  are  from  China,  one  male  and  two 
other  female  students  are  from  Taiwan.  None  of  them  had  previous  experience  with  on-line  courses.  To  keep  the 
survey  anonymous,  pseudonyms  have  been  used  in  this  study. 

On-Line  Course  Format 

The  ET502  - Instructional  Design  course  was  offered  as  a 100%  on-line  course  for  the  first  time  in  the  Fall 
2001  semester.  Two  instructors,  one  American  female  and  one  Asian  male,  co-taught  this  on-line  course  using  a 
web-based  course  management  system  called  Blackboard.  The  interface  of  the  Blackboard  is  shown  in  Figure  1. 
Students  had  to  choose  one  topic  of  interest  and  create  a design  document  and  self-paced  lesson  for  the  topic 
individually.  In  order  to  create  an  on-line  community  among  students,  instructors  randomly  assigned  three  people  to 
form  a group.  All  students  were  informed  in  the  beginning  of  the  semester  that  peer  evaluation  would  be  counted  as 
20%  of  their  final  grade.  The  instructors  also  developed  weekly  mini-lectures  that  synthesized  important 
information  for  topics  that  were  covered  in  the  textbook.  These  mini-lessons  were  then  posted  to  the  Blackboard 
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website.  These  topics  included  writing  a needs  statement,  learner  and  contextual  analysis,  task  analysis, 
instmctional  objectives,  instructional  sequencing,  instructional  strategies,  designing  questions  and  feedback, 
message  design,  and  conducting  formative  and  summative  evaluations. 

Within  each  group,  individuals  were  required  to  work  on  the  first  draft  of  the  weekly  assignments, 
provide/receive  feedback  to/from  their  group  members  (peer  feedback),  revise  their  first  draft  based  on  the  peer 
feedback,  and  post  their  second  draft  on-line.  Following  the  posting  of  this  second  draft,  both  instructors  would  look 
over  their  revised  assignments  on-line  and  provide  feedback  to  each  student.  Students  would  then  revise  their 
second  draft  based  on  the  instructors’  feedback.  These  same  procedures  were  repeated  for  each  assignment  and 
students  would  compile  all  revised  assignments  together  into  a final  design  document.  After  all  sections  of  the 
design  process  were  covered,  students  would  develop  a self-paced  lesson  based  on  the  design  document  that  they 
had  been  developing.  Students  would  then  conduct  a formative  evaluation  to  test  the  self-paced  lesson  to  its  target 
audience.  Finally,  students  submitted  the  final  version  of  the  design  document  and  self-paced  lesson  on-line  at  the 
end  of  the  semester. 


FIGURE  1.  Interface  of  the  Blackboard 
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Materials 

Student  Attitude  Survey.  During  the  last  week  of  the  Fall  2001  semester,  students  completed  a 21 -item 
Student  Attitude  Survey  designed  for  this  study  to  indicate  their  attitudes  toward  this  course  and  the  on-line  learning 
environment.  These  items  were  five-point  Likert -type  questions  that  ranged  from  1 (strongly  disagree)  to  5 (strongly 
agree).  Several  open-ended  questions  dealing  with  students’  likes  and  dislikes  about  this  course,  their  preferred 
delivery  format  for  this  course,  their  suggestions  to  improve  this  course,  and  their  perceptions  toward  on-line 
learning  both  before  and  after  taking  this  course  were  asked. 

Focus  Group  Interview.  After  student  attitude  survey  data  had  been  collected,  coded,  and  analyzed,  the 
authors  found  several  emerged  patterns  that  derived  from  the  data  and  conducted  a focus  group  interview  with  these 
participants. 
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Procedure 

All  subjects  filled  out  the  21-item  Student  Survey  and  answered  open-ended  questions  at  the  end  of  the  Fall 
2001  semester.  A focus  group  interview  was  then  conducted  at  the  beginning  of  Spring  2002  semester.  All  five 
international  students  who  enrolled  in  the  Fall  2001  on-line  course  were  gathered  as  one  group  in  the  department 
conference  room.  Their  responses  were  recorded  and  the  findings  are  reported  in  the  results  section.  The  whole 
interview  lasted  approximately  90  minutes. 

Data  Analyses 

Both  quantitative  (Student  Attitude  Survey)  and  qualitative  (Open-ended  questions  and  Focus  Group 
Interview)  methods  were  used  to  analyze  data  in  this  study. 

Results 

Student  Attitude  Survey 

The  mean  attitude  scores  and  standard  deviations  for  five  international  students’  responses  to  the  21 
statements  on  the  five-point  Likert -type  attitude  survey  administered  after  completion  of  the  on-line  instructional 
design  course  program  are  shown  in  Table  1.  Responses  were  scored  as  5 for  the  most  positive  response  to  1 for  the 
most  negative  response.  The  overall  mean  score  across  the  21  Student  Attitude  Survey  items  was  4.06,  a favorable 
rating  indicating  agreement  with  positive  statements  about  this  course.  The  three  highest-rated  statements  on  the 
survey  were  “I  like  to  see  pictures  of  my  instructors  and  classmates  on  Blackboard”  (M  = 4.80,  SD  = .45);  “I  like  to 
see  the  short  biography  of  my  instructors  and  classmates  on  Blackboard”  (M  = 4.80,  SD  = .45);  and  “I  like  the  mini- 
lectures provided  by  the  instructors”  (M  = 4.80,  SD  = .45).  The  two  lowest-rated  statements  were  “I  would 
recommend  this  on-line  course  to  others”  (M  = 3.00,  SD  = 1.23)  and  “I  would  take  this  course  as  an  on-line  course 
again”  (M  = 3.20,  SD  = 1.48). 

Table  1.  Student  Attitude  Survey  Scores 


Statement 

M 

SD 

1. 

I like  to  see  pictures  of  my  instructors  and  classmates  on  Blackboard. 

4.80 

.45 

2. 

I like  to  see  the  short  biography  of  my  instructors  and  classmates  on  Blackboard. 

.45 

3. 

I liked  the  mini-lectures  provided  by  the  instructors. 

.45 

4. 

I liked  the  “file  exchange”  function  on  Blackboard. 

.55 

5. 

I liked  the  feedback  that  my  instructors  provided. 

.55 

6. 

I spent  more  time  working  on  this  course  than  my  other  courses. 

.55 

7. 

I liked  the  “announcement”  function  on  Blackboard. 

.89 

8. 

I like  to  receive  feedback  from  my  group  members. 

.84 

9. 

I learned  a lot  from  this  course. 

.45 

10. 

I would  like  to  meet  with  my  instructors  and  classmates  face-to-face  some  day. 

.84 

11. 

The  grading  was  fair  in  this  course. 

1.00 

12. 

I liked  the  group  format  in  this  course. 

.84 

13. 

I liked  having  two  instructors  (co-teaching)  in  this  course. 

.84 

14. 

I liked  this  course. 

1.67 

15. 

I liked  the  on-line  environment  of  the  course. 

1.67 

16. 

I like  to  provide  feedback  to  my  group  members. 

.89 

17.  This  course  was  easy. 

.55 

18. 

I liked  the  textbook  that  we  used  in  this  course. 

1.52 

19. 

The  amount  of  the  work  required  was  fair. 

1.34 

20. 

I would  take  this  course  as  an  on-line  course  again. 

1.48 

21. 

I would  recommend  this  on-line  course  to  others. 

1.23 

Total 

4.06 

.52 

Note. 

Reponses  ranged  from  1 (Strongly  Disagree)  to  5 (Strongly  Agree). 
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Student  Perceptions  and  Attitudes  Toward  On-Line  Learning 

Students  were  asked  to  indicate  their  perceptions  toward  on-line  courses  before  taking  ET  502  course  and 
after  taking  ET  502  course.  Their  responses  are  summarized  and  shown  in  Table  2. 

Table  2.  Student  Comments  About  Their  Perceptions 


Student 

Before 

After 

1 

• 

Interesting 

• Would  never  take  on-line  course  again 

• 

Excited 

• This  course  was  too  demanding 

• Lacked  of  instant  interactions 

• Struggled  alone 

2 

• 

Informal 

• Made  a good  choice 

• 

Supplement  for  the  traditional  course 

• Had  to  do  my  best  on  the  projects  and 
provide  feedback  to  peers 

• Received  great  feedback  from  peers  and 
instructor  for  further  revisions 

3 

• 

Naive  attitudes 

• Would  be  very  cautious 

• Spent  plenty  of  time  on  this  course 

• Valued  the  experience 

4 

• 

Something  far  from  my  life 

• Liked  the  on-line  learning  environment 

• 

Extremely  high-tech 

• Interface  was  user  friendly 

• 

Eelt  uneasy 

• Was  easy  to  access  class  materials 

• 

Eeared  of  unknown 

• Had  plenty  of  interactions  with  peers 
and  instructors 

• Eelt  comfortable  of  emailing  the 
instructors  with  questions 

5 

• 

N/A 

• N/A 

When  asked  about  students’  likes  and  dislikes  about  this  particular  on-line  course,  students’  responses  are 
summarized  in  Table  3. 


Table  3.  Student  Attitudes 

Student Likes 

Overall  Convenience 

Comments:  • Could  get  everything  done  at  home 

• Did  not  need  to  worry  about  arriving  in 
class  late,  finding  a parking  apace, 
and/or  driving  to  school  in  cold 
snowing  days 

Elexibility 

• No  fixed  class  meeting  time 

• Did  not  feel  stressful  psychologically 
or  develop  a fearful  attitude  towards 
this  course 

• Could  choose  the  right  time  to  study 
when  highly  motivated 

• Read  course  materials  at  my  own  speed 

• Easy  access  to  the  class  website 


Dislikes 

Interaction 

• The  interactions  among  students  and 
instructors  were  not  enough 

• Lacked  instant  feedback 

Instructional  Design  Concept 

• The  difficult  nature  of  the  course 

• Did  not  know  what  I was  doing 

• The  process  of  completing  the  project 
was  painful 

Others 

• The  rigid  evaluation 

• Technical  problems  (Eile  Exchange) 


Self -regulated  Learning 

• Liked  to  work  on  individual  project 

• Liked  learner-centered  approach 

• Provoked  me  to  learn  more 
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information  by  myself 

• 

When  the  actual  project  was  done,  it 
looks  like  a baby  was  born 

Others 

• 

Liked  to  see  different  topics  and  ideas 
that  other  students  posted  on-line 

• 

Feedback  provided  by  instructors  and 
other  group  members 

• 

The  mini-lectures  are  good  study 
guides  (concise  and  easy  to  read) 

Student  Preferences  Toward  On-Line  Learning 

The  authors  asked  students,  “Do  you  think  the  content  of  this  course  would  be  better  taught  in  a 100%  face- 
to-face  environment,  in  a half  face-to-face  and  half  on-line  environment,  in  a 100%  on-line  course,  or  other 
alternatives?’’  One  student  preferred  to  take  this  course  in  a 100%  face-to  face  environment,  two  students  preferred 
to  take  this  course  as  in  a half  on-line  and  half  face-to-face  environment,  one  student  preferred  to  take  this  course  as 
in  a 100%  on-line  environment,  and  one  student  indicated  no  preference.  Their  responses  and  reasons  given  are 
summaries  in  Table  4. 

Table  4.  Student  Preferences 


Student 

Preferred 

Format 

On-line 

Face-to-face 

1 

100% 

• 

Too  demanding 

• 

Better  discussion  with  peers 

face-to- 

• 

Content  was  too  complex 

• 

Immediate  interactions  with 

face 

• 

No  immediate  feedback 

instructors 

2 

50%/50% 

• 

No  time  and  place  limit 

• 

Questions  could  be  answered 
instantly 

3 

50%/50% 

• 

Flexible  time 

• 

Content  could  be  better  discussed 
in  the  traditional  classroom 

4 

No 

• 

No  difference.  Feel  comfortable 

• 

No  difference.  Feel  comfortable 

preference 

either  way. 

either  way. 

5 

100%  on- 

• 

Access  anytime  anywhere 

• 

N/A 

line 

• 

Read  materials  at  my  own  speed 

• 

Questions  were  answered 
“individually’’  by  email  from  the 
instmctors 

On-Line  Learning  Interactions 

When  asked  students  to  identify  their  level  of  interactions  with  peers  and  instructors  between  the  on-line 
course  and  traditional  classroom  environment,  students’  comments  are  summarized  in  Table  5. 

Table  5.  Student  Interactions 


Student 

On-line 

Traditional 

Overall 

• 

Felt  isolated  and  lonely 

• 

Could  received  immediate  interactions 

Comments: 

• 

Met  with  group  members  in  person 

• 

Usually  sit  quiet,  nod  head,  or  simply 

• 

Interacted  “more”  with  American 

smile  when  agreed. 

students  (intellectually  but  not 

• 

With  more  “what’s  up,  dude”  “how  are 

physically) 

you”  type  of  informal  conversations 

• 

Some  people  did  not  provide 

• 

Could  view  body  language  and/or 

constructive  feedback 

verbal  cues 

• 

Teammates  would  disappeared  for  a 

• 

Would  feel  ashamed  to  express  my 
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while  in  the  cyberspace 
Required  “real”  participation  with  peers 
Direct  interaction  - all  interactions  were 
very  focused 

Less  language  barriers  in  the  on-line 
environment  - no  speaking  and 
listening,  only  reading  and  writing 
Less  misunderstanding  - accent,  feel 
ashamed 

Lacked  language  and  cultural  exchange 


opinions  because  of  my  accent 
Would  improve  my  English  listening 
and  speaking  skills 

Whole  approach:  learning  language  and 
culture  (ex,  American  students  put  their 
feet  on  the  chair,  eat  in  the  classroom, 
and  challenge  their  instructors.  I could 
sense  the  class  atmosphere,  listen  to 
humor  or  jokes  from  the  instructor,  and 
observe  American  teaching/learning 
styles  in  the  classroom.). 


Discussion  and  Educational  Implications 

This  study  investigated  the  cultural  influences  on  the  on-line  learning  environment  and  explored  issues 
considered  for  international  students  who  are  planning  to  take  their  first  on-line  courses.  Instructors  should  also  take 
international  student  needs  into  account  and  develop  appropriate  strategies  to  assist  international  students  who  are 
taking  on-line  courses. 

It  is  interesting  to  note  that  in  the  Student  Attitude  Survey,  the  highest-rated  statements  were  ”I  like  to  see 
pictures  of  my  instructors  and  classmates  on  Blackboard”  and  ”I  like  to  see  the  short  biography  of  my  instructors 
and  classmates  on  Blackboard”.  It  seems  that  international  students  liked  the  idea  of  building  an  on-line  community 
among  peers  and  instructors  in  this  course.  The  two  lowest-rated  statements  were  “I  would  recommend  this  on-line 
course  to  others”  and  “I  would  take  this  course  as  an  on-line  course  again”.  We  believe  that  these  two  statements 
reflected  on  international  students’  attitude  toward  their  first  on-line  learning  course.  As  suggested  by  the  overall 
high  ratings  of  other  survey  items,  the  students  overall  liked  the  course.  Their  neutral  opinions  regarding 
recommending  this  course  to  others  or  taking  this  course  as  an  on-line  course  again,  however,  suggest  that  the  on- 
line experience  could  be  improved  for  international  students. 

In  the  process  of  teaching  this  on-line  course  to  international  students,  reading  their  feedback  and 
assignments,  and  interviewing  them  during  the  focus  group,  the  authors  noted  comments  by  the  students.  The 
authors  would  like  to  provide  some  suggestions  regarding  to  teaching  international  students  in  the  on-line  learning 
environment. 

Initial  Support 

• Instructors  should  provide  more  support  to  increase  international  students’  self  confidence  and  motivation 
in  the  beginning  since  it  was  their  first  time  taking  an  on-line  course.  One  student  stated,  “In  the  on-line 
environment,  the  initial  support  provided  to  students  needs  to  be  strong.  International  students  had 
experienced  traditional  classroom  experience  since  kindergarten.  Anther  student  expressed  that  “I  think 
that  the  grading  for  the  first  two  assignments  were  not  fair  because  of  different  class  format  (on-line  versus 
traditional  face-to-face  format).  The  instructor  should  give  more  time  in  the  beginning  of  the  course  for 
students  to  be  familiar  with  the  interface.” 

Group  Format 

• International  students  would  prefer  to  work  with  a small  group  (preferably  3 or  4 people  in  one  group)  and 
to  give  or  receive  feedback  within  the  group  but  would  prefer  to  work  on  an  individual  project  instead  of  a 
group  project.  One  student  expressed  that  “In  tradition  class  when  we  discuss  a topic  because  I do  not  have 
good  communication  skills  nor  do  not  know  how  to  express  myself  well  because  I am  an  international 
student,  I would  let  American  students  to  express  their  opinions.  But  in  the  on-line  environment,  everyone 
is  equal.  Another  student  mentioned,  “In  the  traditional  class,  if  I have  an  idea  I am  usually  quiet  and  will 
give  in  to  the  idea  of  American  students  in  my  group.  I do  not  know  why.  I just  give  in...  When  working 
individually,  I would  stick  with  my  own  idea  and  just  finish  it.  It’s  more  work  but  it  is  worth  it.” 

• International  students  would  prefer  not  to  be  assigned  to  the  same  group  with  other  international  students  in 
this  on-line  course.  Several  students  stated,  “Our  English  will  never  match  up  with  American  students 
since  English  is  not  our  first  language.  Americans  know  how  to  express  themselves  well  and  have  great 
commutation  skills.  Eor  instance,  when  American  students  conducted  a mediocre  research,  they  could 
express  their  research  to  100%.  On  the  other  hand,  although  Asian  students  understood  the  ins  and  outs  of 
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their  research  studies,  we  could  only  express  50%  no  matter  whether  it  is  in  the  on-line  or  in  the  traditional 
classroom  environment.” 

Peer  Feedback 

• Most  International  students  supported  the  idea  of  providing  feedback  to  their  group  members.  In  that  way, 
they  felt  that  they  were  “forced”  and  had  responsibilities  to  read  the  chapters  and  mini-lectures  thoroughly 
so  they  could  provide  constructive  feedback  to  their  group  members. 

• Most  international  students  suggested  that  providing  and  receiving  feedback  within  a group  was  an  efficient 
way  of  learning  and  the  feedback  were  got  right  to  the  point  and  professional  sounded.  However,  they 
suggested  that  instructors  should  state  criteria  or  rubrics  for  the  peer  feedback  because  some  group 
members  only  provide  superficial  feedback  to  them. 

Self-regulated  Learning 

• They  strongly  supported  the  learner-centered  approach  in  the  on-line  environment  and  agreed  that  students 
should  be  responsible  for  their  learning.  They  indicated  that  in  the  face-to-face  classroom  environment, 
they  only  need  to  sit  quiet  or  nod  their  head  when  agree  with  something.  However,  it  requires  “real” 
participation  and  interaction  in  the  on-line  environment  especially  when  peer  feedback  was  involved. 

Language  and  Culture 

• International  students  felt  that  it  was  easier  for  them  to  communicate  better  with  their  group  members  on- 
line since  there  was  no  English  speaking  or  listening  involved  in  that  environment.  One  student  expressed, 
“I  thought  on-line  course  would  be  easier  for  me  because  my  English  speaking  skill  was  not  good  so  I do 
not  need  to  “speak”  in  the  on-line  environment.  Since  I only  had  to  write  in  this  course,  I had  much  more 
confidence.”  However,  many  of  them  felt  isolated  sometimes  because  they  would  like  to  have  face-to-face 
interactions  with  American  peers  in  order  to  improve  their  English  as  well  as  to  learn  more  about  American 
culture. 

First  Author/Instructor  Reflections 

Through  my  own  observations,  interacting  with  international  students,  being  an  international  student  and 
international  faculty  myself,  I found  this  topic  intriguing.  During  the  process  of  finding  out  the  international 
students’  perceptions  and  attitudes  toward  their  first  on-line  learning  course  experiences,  I might  have  different 
opinions  than  my  American  co-author/instructor  in  this  study  because  of  different  cultural  backgrounds.  I was  born 
and  raised  in  Taiwan,  came  to  the  United  State  ten  years  ago,  and  had  never  taken  any  on-line  courses.  I was 
doubtful  about  the  effectiveness  of  on-line  learning.  I was  wondering  what  students  could  learn  from  on-line 
courses?  What  were  their  motivations  for  taking  on-line  courses?  Would  the  teaching  preparation  time  for 
instructors  be  less  in  the  on-line  environment  than  traditional  face-to-face  classroom?  I would  like  to  express  some 
of  my  observations  and  thoughts  on  these  issues. 

Students  in  Taiwan  and  China  are  eager  to  become  proficient  in  English  - reading,  writing,  listening,  and 
speaking.  Nowadays,  students  start  learning  English  when  they  are  in  the  fifth  grade  in  Taiwan  and  in  seventh  grade 
in  China.  The  newest  trend  in  both  Taiwan  and  China  is  to  start  English  in  the  kindergarten.  As  long  as  English 
teachers  “look”  like  westerners  and  are  English  speakers,  help  the  enrollments  of  schools  because  they  look  like 
“live  models.”  Students  in  Taiwan  or  China  spent  a lot  of  money  to  attend  these  English  conversation  schools  to 
improve  their  English  and  learn  some  American  culture  from  these  instructors.  I question  whether  students  would 
still  be  interested  in  attending  these  courses  and  perform  well  if  they  are  offered  on-line. 

Distance  Education  is  getting  more  popular  and  it  is  a trend  in  higher  education  worldwide  that  no  one  can 
prevent.  Sometime  soon  (or  it  might  have  already  happened),  I believe  that  a lot  of  students  who  are  from  Taiwan  or 
China  will  have  opportunities  to  take  on-line  courses  that  are  offered  by  American  schools  without  physically  being 
in  the  United  States.  This  might  provide  some  advantages  for  some  international  students;  for  example,  they  do  not 
need  to  leave  their  country  and  family,  spend  money  on  rent  and  foods,  deal  with  visa  issues,  etc.  On  the  other  hand, 
they  might  miss  some  great  opportunities  to  learn  about  American  culture  and  English  language,  similar  to  this  study 
suggested.  Can  international  students  learn  about  American  culture  in  the  on-line  environment?  I am  not  so  sure.  I 
believe  that  it  would  be  like  watching  a travel  channel  on  television  and  trying  to  learn  the  culture  via  the  television. 
It  would  also  be  a pity  for  international  students  who  completed  their  whole  degree  program  on-line  to  graduate 
without  knowing  American  culture  and  missing  the  opportunity  to  improve  their  English  skills.  And  vice  versa,  it 
would  be  a shame  to  prevent  Americans  from  learning  more  about  the  cultures  of  China  and  Taiwan. 


236 


For  future  studies,  it  would  be  interesting  to  ask  international  students  to  list  advantages  and  disadvantages  of  taking 

the  whole  degree  program  in  the  on-line  environment.  It  would  be  also  interesting  to  explore  whether 
American  culture  and/or  language  skills  can  be  exchanged  or  transformed  through  the  on-line  environment  among 
international  students  and  American  peers.  The  authors  are  planning  to  conduct  further  research  in  these  areas. 

Second  Author/Instructor  Reflections 

In  some  respects  many  of  the  international  student’s  attitudes  about  their  on-line  experience  are  no  different 
than  that  of  their  American  peers.  Feelings  of  isolation,  frustration  over  the  lack  of  non-verbal  cues,  and  the 
challenge  of  dealing  with  more  verbally  comfortable  (and  often  less  academically  gifted)  team  members  are 
expressed  by  American  students  as  well.  Appreciating  the  self-paced  format,  the  ability  to  take  a class  without 
parking  or  travel  hassles,  and  the  opportunity  to  reflect  and  compose  thought-out  verbal  responses  are  considered 
positive  aspects  of  on-line  learning  by  most  students. 

International  students,  however,  have  additional  challenges  in  each  of  these  areas  because  of  different 
cultural  norms  and  unfamiliar  language.  Future  on-line  courses  can  take  the  challenges  experienced  by  international 
students  into  account  by  describing  the  culture  of  the  on-line  class  up  front.  For  example,  students  can  be 
encouraged,  perhaps  even  required,  to  get  together  in  face-to-face  on  an  occasional  basis.  Special  face-to-face 
sessions  with  the  instructor  of  on-line  courses  can  also  be  offered.  Skills  needed  for  surviving  in  an  on-line  course 
can  be  articulated.  Although  most  of  these  suggestions  address  the  affective  side  of  learning;  these  suggestions  may 
provide  an  additional  layer  of  comfort  for  international  students  new  to  the  on-line  environment. 
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Are  Professors  Ready  for  the  Technology  Age? 
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Nova  Southeastern  University 


Abstract 

Technology  and  the  Internet  have  invaded  the  educational  systems  of  higher  education.  Now,  more  than 
ever,  universities  are  forced  to  change  how  they  deal  with  the  growing  technical  needs  of  these  organizations.  This 
is  particularly  evident  as  for  the  first  time  in  academia;  students  entering  college  may  be  better  equipped 
technologically  than  their  faculty.  Sixty-six  professors  were  surveyed  to  determine  their  preparedness  for  technology 
integration  into  their  curriculum.  The  findings  suggested  that  faculty  did  not  have  the  technical  skills  necessary  to 
conduct  distance  education. 

Background 

Implementing  the  use  of  web-based  technologies  in  teaching  and  learning  requires  fundamental  changes  in 
many  areas  of  an  institution.  Higher  education  institutions  in  Puerto  Rico  are  no  different.  This  study  was  conducted 
to  evaluate  and  assess  the  level  of  technical  expertise  or  competency  of  the  full  time  faculty  in  a private,  nonprofit 
Puerto  Rican  university.  The  research  site  was  established  in  1949  as  a two-year  technical  college.  In  the  year  1991, 
it  became  a four-year  college.  In  2001,  graduate  programs  began  to  be  offered.  Its  academic  programs  lead  to 
baccalaureate  and  associate  degrees  in  liberal  arts,  education,  business  administration,  office  systems,  science  and 
tourism.  It  also  has  programs  on  a transfer  basis  to  continue  toward  bachelor’ s degree,  professional  degrees,  and 
certificates.  This  university  also  offers  master’s  degrees  in  Human  Resources,  Administration  and  Supervision  in 
Education,  Marketing,  Criminal  Justice,  and  Management.  The  main  campus  has  six  university  centers,  spread  out  in 
different  towns. 

Population  Characteristics 

The  student  body  is  composed  of  young  adults  who  mainly  reside  locally  and  also  come  from  adjacent  towns. 
The  majority  of  students  are  from  families  with  low  incomes  and  limited  education.  The  student  body  is  also 
composed  of  youths  and  adults  that  have  recently  graduated  from  high  school,  as  well  as  adults  that  have  entered  the 
job  market  and  need  to  improve  technical  skills  or  desire  an  opportunity  for  self-improvement.  Table  1 shows  the 
amount  of  students  per  year  in  the  program.  On  the  main  campus  a 33%  reduction  of  students  can  be  seen  between 
first  and  second  year  students.  Attrition  continues  throughout  the  academic  years  and  at  both  the  main  campus  and 
the  university  centers.  Further  research  should  look  at  the  extreme  attrition  found  in  the  amount  of  students  per  year. 

Table  1 - Student  Profile 


First  Year 

Second  Year 

Third  Year 

Fourth  Year 

Total 

Main  campus 

2186 

717 

430 

315 

3648 

University 

Centers 

1487 

409 

255 

147 

2298 

Totals 

3673 

1126 

685 

462 

5946 

Table  2 - Faculty  Profile 

Full  Time  on  Campus  Part  Time  on  Campus  University  Centers  Part  Time  Only 

72  143  193 

As  this  institution  recently  went  through  the  licensing  renewing  procedures,  results  of  the  evaluation  suggested 
that  it  distinguishes  itself  from  other  local  universities  in  the  area  in  three  main  areas: 
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Open  to  change  - There  is  a complete  support  system  for  change.  Beginning  with  the  Chancellor,  most  of 
the  professors  that  experiment  with  new  ways  of  doing  something,  always  find  support. 

Professors  are  always  looking  for  new  ways  of  teaching  or  reaching  the  student.  As  the  administration 
supports  change,  the  field  is  open  to  new  methodologies  and  techniques. 

Student  services  - being  a private  institution,  it  is  strange  to  find  so  many  student  services  available.  We 
have  Freshmen  Year  Services  Program,  Professional  Guidance  and  Counseling  Programs,  Advising 
Program,  Mentoring  Program,  Assessment  Program,  Tutorial  Services,  Complementary  Educational 
Services  Program,  Health  Services,  Social  and  Cultural  Activities,  Sports  and  Recreational  Activities, 
Books  and  Supplies,  Veterans  Services,  Student  Activities,  Student  Organizations,  Student  Council, 
Academic  Dean’s  List,  Tutoring,  Work  Study  Program,  Institutional  Scholarship  Programs,  After  School 
Program  for  our  students’  children,  and  opening  in  January,  2002  a new  Pre-School  Center. 


Review  of  the  Literature 

“Communications  technologies  have  the  potential  to  transform  the  educational  process.  [In  many  contexts], 
these  technologies  have  the  power  to  change  the  roles  of  faculty.’’  (Needham,  1986).  Technology  and  the  Internet 
have  invaded  the  educational  system  at  all  levels.  Institutions  frequently  develop  new  programs  or  adopt  new 
paradigms  in  response  to  perceived  changes  in  the  exigency  of  society  as  a whole  or  specific  student  patrons  in 
particular. 

On  July  13-14,  2000,  a group  of  sixteen  higher  education  leaders  gathered  in  Lake  George,  New  York,  to 
participate  in  a symposium.  The  topic  was  "Preserving  Quality  in  Distributed  Learning  Environments.”  This  was  the 
third  of  the  Pew  Symposia  in  Learning  and  Technology.  The  purpose  of  these  symposia  was  to  conduct  an  ongoing 
national  conversation  about  issues  related  to  the  intersection  of  learning  and  technology.  During  the  presentations, 
data  was  presented  that  mentioned,  according  to  Carole  Cotton  of  CCA  Consulting,  a market  research  firm,  that 
ninety-four  percent  of  all  colleges  and  universities  are  either  currently  (63%)  or  planning  to  be  (31%)  engaged  in 
distance  and  distributed  learning.  Some  believe  that  this  extraordinary  growth  is  surpassing  the  existing  quality 
assurance  capacities  of  accrediting  agencies.  Others  counter  that  distance  learning  is  a long-established  form  of 
higher  education  and  that  quality  assurance  practices  for  distance  education  are  essentially  the  same  as  those  used 
for  traditional,  on-campus  education.  Regardless,  the  arrival  of  distance  and  distributed  learning  has  raised  numerous 
questions  about  quality  assurance.  How  do  established  distance  learning  institutions  ensure  quality?  What  more 
needs  to  be  done?  How  do  quality  assurance  agencies  view  the  distinction  between  on-  and  off-campus  teaching  and 
learning  (Twigg,  2000)? 

Dissatisfaction  with  the  educational  status  quo,  both  from  inside  and  outside  the  university,  has  increased  the 
pressure  on  higher  education  to  reevaluate  its  traditional  educational  practices.  Higher  education  is  facing  the 
challenges  of  the  rapid  development  of  information  technology,  increased  accountability  for  quality  education, 
competition  from  online  institutions,  rising  costs,  and  changing  student  population  (Ehrmann,  1995;  Guskin,  1994; 
Handy,  1998;  Owston,  1997). 

Consequently  some  universities  and  colleges  are  considering  several  restructuring  solutions.  Some  of  these 
solutions  target  educational  practices:  modifications  in  mission  statements  urging  better  preparation  of  students  for 
work  and  life  skills  (Handy,  1998);  curricula  re-design  with  emphasis  on  measurable  learning  outcomes  (Diamond, 
1997);  a redefinition  of  pedagogical  goals  toward  a more  learner-centered  pedagogy  (Collis,  1998);  and  heavy 
investment  in  technological  innovations,  such  as  the  Internet  (Owston,  1997).  Of  theses  solutions,  the  latter  has 
received  much  attention  in  higher  education  literature.  At  the  research  site,  as  in  many  other  universities, 
administrators  have  high  expectations  that  the  potential  of  the  Internet  can  be  realized  to  better  serve  higher 
education  clientele  (Duchastel,  1997)  and  thereby,  address  some  of  its  restructuring  concerns  (Ehrmann,  1997a; 
Guskin,  1994;  Owston,  1997). 

Reasons  for  such  expectations  arise  from  several  factors.  Lirst,  the  Internet’s  communication  capabilities  and 
its  ability  to  transmit  and  store  vast  amounts  of  information  hold  great  promise  for  the  restructuring  of  teaching  and 
learning  practices.  Already  two  of  its  most  popular  services,  electronic  mail  (e-mail)  and  the  World  Wide  Web 
(WWW)  are  changing  teaching  and  learning  practices  since  “teacher  and  text’’  are  no  longer  perceived  as  the  only 
resource  available  to  learner.  Today,  e-mail  allows  for  access  to  global  peers,  colleagues,  experts,  and  researchers. 
Similarly,  the  WWW  brings  a variety  of  multi-media  resources  to  the  fingertips  of  students  at  a speed  and  quality  of 
transmission  never  experienced  before  though  networked  computer  technology. 
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The  1999  Campus  Computing  Project’s  annual  national  survey  of  information  technology  in  higher  education 
reported  that  many  higher  education  institutions  were  providing  more  services  through  the  WWW.  For  example, 
more  than  half  (69.5%)  of  the  institutions  responding  provided  online  undergraduate  applications  on  their  Web  sites. 
More  than  three-fourths  (77.3%)  made  the  course  catalog  available  online.  A quarter  (25%)  of  the  institutions  made 
library -based  course  reserve  reading  available  online  and  almost  half  (46.5%)  offered  one  or  more  full  college 
courses  online  via  the  Internet  and  the  WWW  (Campus  Computing  Project,  1999). 

Second,  the  Internet  allows  for  greater  adaptability  of  instructional  methods  to  different  learning  environments 
making  it  ideal  for  distributed  learning  systems  capable  of  accommodating  the  academic  needs  of  a diverse  adult 
student  population.  Third,  traditional  classrooms  in  higher  education  have  been  criticized  for  inadequate  training  of 
students  for  the  work  place.  The  boredom,  seeming  lack  of  personal  attention,  outdated  class  material,  and 
inadequate  sensitivity  to  a diverse  population  are  some  of  the  issues  that  have  been  questioned  (Gardiner,  1997; 
Handy,  1998). 

To  further  complicate  matters,  it  has  been  argued  that  many  students  graduate  without  mastering  core  skills 
(Diamond,  1997).  Meanwhile,  state  of  the  art  technologies  are  being  adopted  by  corporations  which  are  spending 
billions  of  dollars  annually  on  formal  training  to  make  curricula  and  teaching  methods  more  relevant,  interesting, 
and  modern  (Mauch,  1998).  Such  possibilities  are  forcing  the  traditional  establishment  and  administration  of  higher 
education  but  to  jump  onto  the  technological  bandwagon  (Sherrit  & Basom,  1997).  Similarly,  Rossner  and  Stockley 
(1997)  summed  up  the  situation  as  one  of: 

...  increasing  public  access  to  information  technology,  government  funding  priorities  for  initiatives  that 
support  the  development  of  distributed  learning  networks,  increasing  competition  for  students  who  no 
longer  need  to  be  present  on  a university  campus,  and  the  need  to  provide  cost-effective,  high  quality 
education  to  greater  numbers  of  students,  (p.333) 

At  the  research  site,  an  intellectual  debate  has  been  brewing.  Administrators  have  to  contend  with  the  pressure 
to  integrate  technology  as  a means  of  attracting  student  applications  and  faculty  either  rejecting  its  imposition  onto 
their  teaching  practices  or  seek  support  for  technology  integration  rewards,  and  recognition  for  the  time  and  effort 
that  goes  into  such  efforts.  Green  (1999)  reported  that  the  1999  survey  of  college  faculty  from  UCLA’s  Higher 
Education  Research  Institute  (HERI),  which  provides  the  first  large-scale  data  on  some  aspects  of  the  way  college 
faculty  use  - and  do  not  use  - IT  resources  documented  that  “keeping  up  with  information  technology’’  is  a major 
source  of  stress  for  fully  two-thirds  of  the  professorate  (p.3). 

Background  and  Problem 

Though  greeted  with  enthusiasm  by  the  university  community  at  large,  technological  integration  remains 
controversial.  Opponents  caution  against  hasty  and  ad  hoc  technology  integration  into  daily  practices  of  teaching 
and  learning,  specifically  without  taking  into  consideration  faculty  and  student  willingness  and  cooperation  in  the 
integration  efforts  (Noble,  1998;  Schrage,  1998;  Turoff,  1997;  Young,  1998). 

The  Internet  is  being  heralded  as  a versatile  medium  to  support  learner-centered  educational  practices.  It  is 
being  promoted  as  a technological  innovation  that  can  reduce  mundane  time-consuming  tasks  of  faculty  workload. 
However,  there  is  evidence  for  discontentment  and  controversial  opinions  regarding  the  above  from  both  faculty  and 
student  bodies  that  only  time  and  empirical  research  could  serve  to  dispel. 

Many  universities  are  making  substantial  investments  in  new  technologies  for  teaching  purposes.  However, 
although  there  has  been  widespread  adoption  of  new  technologies  for  teaching  in  the  last  few  years,  they  have  not 
brought  about  major  changes  in  the  way  teaching  is  organized  and  delivered  (Bates,  1997). 

Why  use  technology?  It  all  depends  who  you  ask.  Politicians,  university  presidents,  keynote  speakers  at 
conferences  from  industry,  education  and  government  and  teachers  themselves  offer  a number  of  different  reasons  to 
justify  the  use  of  technology.  Bates  (1997)  mentions  fourreasons: 

To  improve  access  to  education  and  training 
To  improve  the  quality  of  learning 
To  reduce  the  costs  of  education 
To  improve  the  cost-effectiveness  of  education 

Some  of  these  reasons  seem  conflicting.  More  students  might  be  served  by  the  institution  through  using 
technology:  more  means  worse  quality  and  more  work  for  faculty.  The  initial  set  up  cost  anticipates  delving  deep 
into  the  university’s  pocket.  Eor  the  same  dollar  amount  learning  effectiveness  can  be  increased,  or  more  students 
can  be  taught  to  the  same  standard  for  the  same  level  of  investment.  This  seems  a great  goal  to  strive  for.  Twigg 
(2000),  states,  “making  use  of  new  technologies  to  reduce  the  cost  of  instruction  requires  a fundamental  shift  in 
thinking.’’  It  requires  challenging  the  assumption  of  the  current  instructional  model:  face-to-face,  synchronous 
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learning.  Colleges  and  universities  must  focus  on  producing  effective  use  of  resources  - faculty,  time  and 
technology,  to  increase  student  learning.  In  order  to  reduce  costs,  without  hurting  the  quality  of  instruction,  the 
university  must  move  away  from  traditional  management  of  resources,  contact  hours,  and  time.  (Twigg,  2000; 

Bates,  1997). 

Twigg  (2000)  states  that  “A  substantial  amount  of  an  institution’s  faculty  members  mu  st  have  an 
understanding  of  and  some  experience  with  integrating  elements  of  a computer-based  instruction  into  existing 
courses.”  This  is  paramount.  When  the  organization  purchases  technology,  it  is  imperative  to  involve  professors, 
administrators,  support  personnel  and  students.  Some  faculty  members  may  have  a great  enthusiasm  for  redesigning 
their  courses,  but  have  little  experience  in  this  area.  Twigg  (2000)  mentions  that  experts  have  stated  that  13  to  15 
percent  of  the  faculty  should  constitute  critical  mass.  More  specifically,  13  to  15  percent  of  faculty  is  needed  to 
begin  smaller  instructional  technologies  units  and  thus  satisfy  integration  needs  in  order  to  create  a change  reaction 
moving  towards  large-scale  redesign  efforts.  It  is  clear  then,  that  universities  must  begin  by  engaging  faculty, 
training  them  and  providing  the  professional  development  needed.  Again,  the  need  to  acquaint  faculty  to  the 
organization’s  technical  efforts  is  crucial.  “One  of  the  main  challenges  of  making  technology-based  teaching  a core 
function  is  to  extend  its  implementation  from  a relatively  small  number  of  enthusiasts  and  early  adopters  to  the  main 
body  of  the  teaching  force”  (Bates,  1997).  Teaching  with  technology  is  not  an  inferred  process,  appropriate  training 
must  be  emphasized.  Faculty  must  go  through  a four-step  sequence  to  develop  skills  in  using  technology  in  teaching. 
First,  they  must  understand  the  importance  of  using  technology  for  teaching.  They  need  some  basic  understanding  of 
teaching  and  learning  processes  and  different  approaches.  Third,  they  must  comprehend  the  roles  that  technology 
can  play  in  teaching  and  how  this  changes  the  organization  of  content.  Fourth,  they  should  know  how  to  use  a 
particular  piece  of  technology.  (Bates,  1997). 

Methodology/Results 

The  primary  goal  of  this  research  study  was  to  quantify  the  level  of  technical  expertise  or  competency  of  the 
full-time  faculty  through  self-report.  It  was  determined  that  the  full-time  professors  would  provide  more 
generalizable  data,  due  to  the  fact  that  they  are  involved  in  the  university  growth.  A technical  skills  survey  was 
given  to  all  full-time  faculty  (n=72).  The  response  rate  was  excellent  (92%)  as  66  replied.  Data  was  collected  during 
a yearly  faculty  meeting  at  the  beginning  of  the  2001  academic  year.  Variables  that  were  assessed  included: 
department,  years  of  experience,  and  technical  skill  level.  The  technical  skills  reported  by  faculty  were:  Word 
processing.  Presentations,  Using  Laptop  with  Projector,  Email,  Databases,  WWW,  Discussion  Board,  Spreadsheets, 
Distance  Education,  Statistics,  Chats,  HTML,  Mailing  Lists,  and  Graphic  Design.  Information  was  also  collected 
regarding  prior  experience  with  distance  education  and  the  level  of  experience  with  distance  education  based  on  the 
following  three  categories:  some  resources  on  the  web,  web  enhanced  (additional  resources)  and  completely  on-line. 

The  results  are  divided  into  three  sections.  The  opening  section  involves  the  primary  analysis  of  the  data 
(descriptive  statistics  for  the  overall  sample,  departmental  affiliation,  and  years  of  experience).  The  next  section 
involves  the  secondary  analysis  of  the  data,  specifically,  differences  by  departmental  affiliation  and  also  years  of 
experience  and  its  relationship  to  technical  skill  level.  Einally,  previous  experience  with  distance  education  will  be 
highlighted. 

Descriptive  Statistics 

The  overall  sample  distribution  of  mean  technical  skills  are  listed  below  in  Eigure  1.  This  figure  represents  the 
entire  sample’s  responses  to  the  technical  skills  portion  of  the  survey.  This  evenly  distributed  bell-shaped  curve  was 
indicative  of  typical  performance. 
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FIGURE  1 - Distribution  of  Technical  Skills  Scores 
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In  Figure  2,  bars  represent  each  technical  skill  and  the  corresponding  average  for  the  entire  sample.  E-mail 
(3.89)  and  word  processing  (3.82)  were  self-reported  as  being  the  highest  level  of  competency.  Graphic  design 
(1.84)  and  mailing  list  (1.82)  were  ranked  at  the  bottom  in  terms  of  technical  skill  level. 

FIGURE  2 - Technical  Skills  and  Corresponding  Averages 
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The  following  table  (Table  3)  includes  the  results  of  the  fourteen  technical  skills  for  the  entire  sample 
(n=66).  These  descriptive  statistics  include  means  and  standard  deviations.  As  displayed  by  the  lowest  standard 
deviations,  the  sample  shared  the  greatest  similarity  regarding  mailing  lists  (also  referred  to  as  list-servs). 

Table  3 — Technical  Skills,  Means  and  Standard  Deviations 


Skills 

N 

M 

SD 

EMAIL 

66 

3.89 

1.27 

WP 

64 

3.82 

1.26 

PRESENTA 

65 

3.46 

1.42 

LAPTOP 

64 

3.20 

1.62 

EXCEL 

63 

2.63 

1.54 

DATABASE 

64 

2.60 

1.34 

WEB 

63 

2.36 

1.40 

CHAT 

62 

2.04 

1.48 

STATS 

63 

2.00 

1.29 

DE 

61 

1.96 

1.25 

DISCUSS 

63 

1.92 

1.33 

HTML 

63 

1.84 

1.25 

GRAPHICS 

63 

1.84 

1.23 

LISTS 

62 

1.82 

1.20 

As  seen  in  Figure  1,  the  overall  sample  generated  a highly  consistent  and  typical  bell -shaped  curve. 
However,  when  the  departments  were  analyzed  separately,  significant  differences  were  exposed.  The  following 
graph  (Figure  3)  represents  the  sample  breakdown  by  department.  Depart  ments  were  somewhat  evenly  distributed 
which  was  hoped  to  produce  stable  inferences.  However,  please  note,  due  to  the  relatively  small  amount  of  faculty  in 
Tourism,  faculty  in  this  department  were  not  included  in  the  statistical  analyses. 

FIGURE  3 - Samp  le  Breakdown  by  Department 
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Results  of  the  highest  and  lowest  self-rated  technical  skills  by  department  are  listed  below  in  Table  4.  Liberal 
Arts  and  Business  both  stated  email  as  the  highest  technical  level.  As  seen  in  the  overall  sample,  the  departmental 
affiliation  demonstrated  consistent  findings. 

Table  4 - Highest  and  Lowest  Technical  Skills  by  Department 

Department  Highest  Lowest 

Liberal  Arts  email  (3.12)  Mailing  lists  (1.27) 

Business  email  (4.38)  Art/Drawing/Graphic  Desgn  (1.94) 

Science  & Technology  WP  (4.50)  HTML  (1.70) 

Averages  were  computed  for  all  technical  skills  by  each  department  and  statistically  significant  differences 
were  found.  These  departments  and  their  corresponding  averages  are  listed.  Liberal  Arts  and  Languages:  (1.80) 
Science  & Technology:  (2.58),  Business  Administration:  (2.97).  Cohen’s  d effect  sizes  statistics  (1988)  were 
computed  to  compare  overall  differences  among  departments.  This  statistic  quantifies  the  difference  between  means, 
and  does  so  in  standard  deviation  units.  For  example,  a Cohen’s  d of  .5  is  interpreted  as  two  groups  being  half  a 
standard  deviation  apart.  Cohen  (1988)  stated  that  anything  above  .20  is  small,  .50  is  medium,  and  .80  is  large. 
Differences  regarded  as  “large”  were  found  when  conducting  this  comparative  statistic. 

This  apparent  variation  in  the  overall  scores  of  departments  was  further  uncovered  through  an  analysis  of  each 
technical  skill.  Based  upon  departmental  scores  of  the  14  categories  of  technical  skills,  7 were  found  significant  at 
the  .01  level  and  4 were  significant  at  the  .05  level.  Those  significant  at  the  .01  level  were:  Word  processing, 
presentations,  laptop,  email,  database,  excel,  and  chat.  At  the  .05  level  were:  WWW,  stats,  html,  and  mailing  lists. 
Differences  among  the  departments  were  so  great  in  technical  skills  that  only  3 skills  did  not  produce  statistically 
significant  results.  These  were  discussion  boards,  distance  education  and  graphic  design.  There  were  great 
disparities  in  skill  levels  based  on  department  affiliation.  This  is  obviously  a concern  to  the  university  and  therefore; 
any  faculty  development  initiatives  should  involve  a further  investigation  of  this  apparent  variation  of  technical 
skills  among  departments. 

Table  5 - Years  of  Experience 


Years  of  Experience 
0-5 
6-10 
11-15 
16-20 
21-1- 


Highest 
email  (4.11) 
WP/email  (4.40) 
WP/email  (4.00) 
email  (3.75) 
WP  (3.70) 


Lowest 

Mailing  Lists  (2.00) 
Mailing  Lists  (2.20) 
HTML  (1.60) 
Discussion  Board  (1.25) 
HTML  (1.45) 


The  breakdown  by  years  of  experience  did  not  produce  statistically  significant  differences.  In  essence,  the 
groups  shared  similar  characteristics  regarding  technical  skills  levels.  Again,  these  groupings  shared  responses  that 
were  similar  to  the  overall  sample. 

Correlations  between  Years  of  Experience  & Technical  Skill  Level 

While  groups  based  on  years  of  experience  alone  did  not  differ  significantly  significant  differences, 
correlations  (Spearman’s  rho)  analyzing  years  of  experience  and  technical  skill  levels  did  produce  statistically 
significant  results.  Chat  and  HTML  offered  negative  correlations  at  the  p<.01  level.  Therefore,  faculty  with  less 
years  of  experience  had  higher  ratings  in  these  areas.  Conversely,  faculty  with  more  years  of  experience  had  lower 
ratings  in  these  areas.  These  findings  were  very  consistent  with  the  literature  and  the  researchers’  expectations.  It 
appears  that  younger  faculty  have  more  familiarity  with  these  higher  order  technical  skills  than  older  faculty.  Such 
findings  point  to  the  need  to  include  all  memb  ers  of  the  faculty  in  terms  of  professional  development. 
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Distance  Education  Experience 

Information  was  collected  to  meet  Twigg’s  theory  of  critical  mass.  On  the  following  survey  question  dealing 
with  “Experience  with  Distance  Education’’,  the  following  responses  were  stated:  yes  (n=13)  no  (n=51).  Also,  of 
those  that  stated  having  previous  experience  with  distance  education  (n=13),  based  on  the  three  categories,  the 
responses  are  shown  below  in  Table  6. 

Table  6 - Distance  Education  Categories/ Levels 


Level 

Descrintion 

Resnonses 

1 

Some  resources  on  the  WWW 

7 

2 

Web -enhanced 

5 

3 

Eully  on-line 

1 

This  finding  can  be  interpreted  as  such;  20%  of  faculty  surveyed  had  previous  experience  with  Distance 
Education.  This  meets  (and  exceeds)  Twigg’s  recommendation  necessary  for  critical  mass.  While  this  finding  is 
interesting,  it  only  hints  that  a move  to  distance  education  would  be  successful.  This  university  is  beginning  to 
involve  itself  more  in  this  genre,  however,  based  on  the  low  technical  levels  of  the  faculty,  this  finding  is  still  left  to 
conjecture. 

Conclusions 

The  findings  of  this  research  convey  an  apparent  disproportion  of  technical  skills  among  faculty. 
Departmental  affiliation  provided  the  greatest  variation  in  technical  skills.  This  points  to  the  need  for  greater 
semblance  of  technical  skills  among  faculty  at  this  institution.  In  a perfect  world,  all  faculty  in  an  academic 
institution  would  share  common  technical  skills.  Yet,  as  reflected  in  society,  commonalities  regarding  technical 
competency  is  atypical.  The  “digital  divide’’  apparent  in  society  is  also  reflected  in  academia. 

Based  on  the  findings  of  this  study,  the  critical  mass  required  to  effectively  integrate  technology  was  evident. 
Yet,  the  variations  in  skills  level  provided  a great  disparity  among  faculty.  Basically,  faculty  appear  interested,  albeit 
at  different  levels.  However,  this  is  not  acceptable  in  higher  education  where  standards  are  typically  required  to 
ensure  equal  outputs  in  terms  of  students  objectives,  etc.  Thus,  to  level  the  playing  field,  faculty  development  should 
look  to  raise  the  technical  levels  of  those  determined  to  be  “weak”. 

While  this  institution  is  interested  in  conducting  distance  education  in  the  near  future,  the  faculty 
preponderance  of  lower  level  skills  is  a concern.  As  expected,  the  Science/Technology  and  Business  departments 
scored  higher  than  Liberal  Arts.  However,  the  technical  skills  ranked  highly  were  typically  “lower-level”  skills 
(email,  word  processing).  While  the  entire  sample  shared  similar  low  levels  of  technical  skill  competencies  (email, 
word  processing),  there  was  an  overall  lack  of  the  higher  order  skills  typically  used  in  distance  education  (html, 
mailing  lists). 

Further  Research 

The  conclusion  of  this  study  also  gives  direction  to  further  research.  These  findings  demonstrated  weak  levels 
of  technical  competencies  which  point  to  the  need  for  better  preparation  regarding  the  technical  skills  and  the 
integration  of  these  skills  into  instruction.  As  purported  by  Bates  (1997)  faculty  must  go  through  a four-step 
sequence  to  develop  skills  in  using  technology  in  teaching.  They  must  understand  the  importance  of  using 
technology  for  teaching.  They  need  some  basic  understanding  of  teaching  and  learning  processes  and  different 
approaches.  They  must  comprehend  the  roles  that  technology  can  play  in  teaching  and  how  this  changes  the 
organization  of  content.  Einally,  they  should  know  how  to  use  a particular  piece  of  technology.  A further 
investigation  determining  the  level  of  understanding  regarding  these  principles  should  be  completed  to  greater 
understand  the  process  of  integrating  technology  into  teaching.  The  inclusion  of  these  approaches  in  professional 
development  would  be  an  effective  start  to  ameliorate  such  deficiencies  of  faculty. 
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Abstract 

The  School  of  Education  at  Indiana  University  South  Bend  endeavors  to  develop  committed  educators  to 
meet  the  needs  of  a diverse  population.  Reflective  practice  is  one  of  the  cornerstones  used  in  teacher  development. 

In  this  paper,  we  explore  the  use  of  case  studies  to  assist  novice  teacher  educators  to  reflect  on  issues  associated 
with  the  integration  of  technology  into  their  teaching  practices  and  patterns. 

Introduction 

The  School  of  Education  at  Indiana  University  South  Bend  endeavors  to  develop  committed  educators  to 
meet  the  needs  of  a diverse  population.  Reflective  practice  is  one  of  the  cornerstones  used  in  teacher  development. 
Case  studies  provide  one  avenue  of  developing  reflective  practice.  Cases  can  provide  an  opportunity  to  experience 
issues  similar  to  those  they  might  encounter  as  they  move  from  pre -service  teachers  to  novice  teachers  in  the 
classroom.  The  case  experience  can  provide  a structure  for  them  to  use  as  they  engage  in  reflective  problem  solving 
and  assess  their  own  performance  in  the  classroom.  In  this  research  project,  we  explored  the  use  of  case  studies  to 
assist  pre -service  teacher  educators  to  reflect  on  issues  associated  with  the  integration  of  technology  into  their 
teaching  practices  and  patterns. 

Background 

Research  from  this  project  was  primarily  focused  on  case  studies  as  a tool  for  reflective  practice.  To  a 
lesser  extent,  we  also  addressed  the  role  of  novice  and  expert  in  approaches  to  learning.  In  her  review  of  case 
studies  in  teacher  education,  Merseth  (1996)  observes  that  cases  should  be  real,  require  research,  and  provide  a 
source  for  discussion  by  users.  One  of  the  emerging  formats  for  case  studies  identified  by  Ertmer  and  Russell  (1995) 
was  fiction  ground  in  real-world  practice  end  experience  that  focused  on  key  issues  or  a central  theme.  Exploration 
of  fictionalized  cases  to  help  students  develop  their  problem  -solving  skills  and  to  create  links  between  theory  and 
practice  have  been  extant  for  some  time  (Kinzie,  Hrabe  & Larsen,  1998;  McNergney,  Herbert  & Eord,  1993;  Cooper 
& McNergney,  1995;  Julian,  Kinzie  & Larsen,  2001) 

We  were  also  interested  in  touching  upon  how  pre -service  teachers  perceived  the  expert -novice  connections 
with  the  context  of  the  case.  We  turned  to  research  in  instructional  design  (ID)  practice  in  the  1990s  that  explored 
the  dynamics  of  the  profession  through  the  eyes  of  those  deemed  experts.  Building  upon  and  confirming  prior 
research  into  professional  design  practice,  Julian  (2001)  established  that  expert  instructional  designers  often 
integrate  professional  knowledge  with  prior  experience  to  assess  situations  and  consider  potential  solutions.  Experts 
recognize  that  situations  are  commonly  influenced  by  multiple  factors  and  therefore  interpret  problems  as  ill 
defined,  spending  more  time  on  front-end  analysis.  They  often  use  mental  models  or  templates  for  the  analysis  and 
solution  generation,  and  recommend  a variety  of  interventions  (Rowland,  Parra  & Basnet  1994).  The  ability  to  solve 
complex  problems  and  identify  solutions  in  this  way,  however,  does  not  ensure  successful  practice.  The  artistic  and 
intrinsic  elements  of  "professional  practice  have  at  least  as  much  to  do  with  finding  the  problem  as  with  solving  the 
problem"  (Schon,  1987,  p.l8).  The  successful  professional  has  the  ability  to  build  a sense  of  community  among 
stakeholders;  creating  an  environment  of  joint  exploration,  enabling  clients  and  colleagues  to  participate  in  the 
process  from  the  discovery  of  the  instructional  need,  to  an  analysis  that  leads  the  team  on  a path  towards  the  best 
possible  solution  (Pitnik,  1995).  We  hoped  to  incorporate  these  components  in  the  case  developed  for  this 
investigation. 
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Method 

Case  studies  were  introduced  into  a technology-related  course  offered  by  the  lUSB  School  of  Education 
during  the  fall  of  2002  to  further  reflective  practice.  Prior  to  the  class  case  analysis,  a formative  evaluation  was 
conducted  for  purposes  of  usability  testing  the  case  in  a separate  course.  The  researcher  and  instructors  for  the 
courses  reviewed  the  procedures  for  the  case  study.  Students  in  the  School  of  Education  were  invited  to  participate 
by  exploring  the  case,  discussing  the  ideas  with  other  members  of  the  class  and  posting  responses  to  questions  about 
the  case  in  a designated  forum  in  Oncourse.  The  lUSB  School  of  Education  uses  Oncourse,  a proprietary,  online 
course  management  application  developed  by  Indiana  University,  for  Web-based  instruction  and/or  instructional 
support  (Indiana  University,  2000).  Case  questions  were  formulated  based  on  identification  of  issues,  values, 
knowledge,  actions  and  the  consequences  of  actions  (Kinzie,  Hrabe  & Larsen,  1998;  McNergney,  Herbert  & Lord, 
1993;  Ertmer  & Quinn,  2003).  Although  students  are  involved  in  the  study,  they  are  as  participants  and  not 
researchers  for  the  study.  The  participation  is  voluntary.  Participation  grades  were  assigned.  Alternative  activities 
were  provided  to  earn  participation  points  for  students  who  elected  not  to  participate  in  the  case,  participants 
involvement  required  a commitment  of  approximately  seven  hours  to  the  research.  Students  read  case  related 
materials  in  advance  of  class  (approximately  2-3  hours).  Participant  involvement  in  the  study  lasted  three  weeks  in 
two  different  sessions.  The  first  session  was  a 75 -minute  training  period  using  Terry  Kirkland.  The  participants 
received  print  copies  of  the  case  and  were  prepared  for  discussion.  The  second  case,  Becky’s  Time  Quest,  was 
discussed  two  weeks  later.  The  case  was  posted  to  Oncourse  the  weekend  prior  to  the  class  meeting.  Questions 
were  posted  on  the  Qncouse  discussion  forum  for  response.  In  class,  participants  discussed  the  case  within  small 
groups  of  three,  and  then  posted  responses  to  the  questions  as  a team  here  and  after  referred  to  as  “respondent”. 

Since  the  responses  posted  were  a group  consensus,  individual  identifications  have  been  removed.  Although  they 
had  the  option  to  post  responses  in  the  forum  for  one  week  following  the  research  activity,  all  the  teams  finished 
during  allotted  class  period  of  2.5  hours.  Eacts/Issues/Problems/Opportunities:  What  knowledge  or  information  do 

you  already  possess  about  instruction  in  this  situation?  What  additional  information  do  you  need  to  know  to 
address  Becky’s  situation? 

• Perspectives : Who  are  the  stakeholders  in  this  case  scenario?  What  perspectives  do  they  (or  might  they) 
bring  to  this  situation? 

• Knowledge:  Can  you  describe  how  an  expert  instructor  might  successfully  facilitate  this  unit?  Please 
consider  responses  from  the  perspectives  of  instructional  methods,  management,  learners  with  special 
needs,  integration  of  subjects  and  invisible  use  of  technology. 

• Actions : Describe  potential  solutions  to  the  issues  identified.  Can  you  describe  what  a novice  teacher  might 
do  in  the  same  situation  and  what  challenges  he  or  she  might  encounter?  Please  reflect  upon  the  following 
aspects:  issues  concerning  classroom  management,  issues  concerning  scheduling,  student  use  of  video 
technology,  student  use  of  software  for  design  and  promotion  (storyboarding,  budgeting,  etc.),  colleagues, 
administration,  assessment  and  standardized  testing  requirements. 

• Consequences : Consider  and  describe  the  possible  consequences  of  each  solution. 

• Recommendations:  Make  a recommendation  for  action  after  weighing  the  advantages  and  limitations  of 
each  solution. 

• Other  Do  you  think  this  case  will  help  you  transition  fro  m novice  to  an  expert  in  the  classroom? 

Student  responses  were  assessed  and  evaluated  using  qualitative  methodology.  In  addition  to  written  case 

responses,  student  response  to  semi -structured  interviews  will  be  explored  using  qualitative  analysis  methods  to 
identify  patterns,  etc.  Analysis  techniques  will  be  refined  and  new  avenues  may  be  explored  as  the  case  is  developed 
during  the  spring  semester  of  2002. 

Results 

A total  of  ten  (10)  students  elected  to  participate  in  the  case;  one  of  which  was  used  for  formative 
evaluation.  Since  the  class  mix  was  dependent  on  enrollment,  and  there  was  not  a specific  focus  on  the  balance  of 
male/female  student  population.  In  this  instance,  all  participants  were  female.  All  case  participants  were  pursuing 
careers  in  special  education.  No  minors  were  involved  in  the  study.  The  results  are  reported  in  general  terms  but 
individual  quotes  were  pulled  for  purposes  of  elucidation,  clarification,  or  emphasis. 

Facts/Issues/  Problems/Opportunities:.  All  four  respondents  [teams]  identified  the  majority  of  issues. 
Among  them,  all  issues  were  identified.  One  team  introduced  their  issues  statement  as  follows: 

Being  in  education  we  understand  that: 

• enthusiasm  sometimes  overpowers  practicality 

• it  is  often  easier  to  learn  than  to  teach 
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• some  ideas  seem  really  fun,  but  are  a lot  more  work  than  meets  the  eye 
One  facet  of  case  analysis  considers  what  information  is  yet  needed  to  better  assess  the  situation. 
Suggestions  from  participants  included  more  data  on  the  technology  background  of  Becky  and  her  students, 
substantially  more  background  on  the  special  needs  of  her  students,  adding  that  it  would  be  "‘helpful  to  know  if  she 
collaborated  with  special  educators  of  her  students  prior  to  implementing  the  project”.  One  respondent  observed 
some  positive  aspects  to  Becky’s  experience  as  well,  observing  that: 

Becky  was  excited  about  teaching. 

Technology  was  being  used  in  the  classroom. 

Some  students  who  were  not  performing  previously  are  now  participating. 

Perspectives : Some  teams  identified  stakeholders  in  greater  depth  and  details  than  others.  Two  respondents 
elected  to  define  nearly  all  of  the  characters  by  function  rather  than  name.  One  respondent  [team]  opted  to  align  the 
descriptions  of  various  teachers  and  parents  in  the  case  into  stakeholders  who  supported  the  protagonist  efforts  and 
those  who  did  not.  Each  respondent  identified  the  major  stakeholders,  a couple  respondents  identified  additional 
important  stakeholders,  collectively,  the  respondents  identified  all  the  stakeholders.  Participants  demonstrated  a 
clear  understanding  that  stakeholders  extended  beyond  the  immediate  characters.  Three  of  the  four  respondents 
identified  the  school  administration  as  stakeholder,  describing  the  principal  as  ‘‘impressed  with  integration  of 
technology"  and  ‘‘supportive  of  her  efforts  to  integrate  technology  into  the  classroom” . One  even  discussed  the 
project  from  the  perspective  of  the  school  corporation  in  which  Becky’s  school  resides. 

Knowledge:  Describe  how  an  expert  instructor  might  successfully  facilitate  this  unit.  They  considered 
responses  from  the  angles  of  instructional  methods,  classroom  management,  students  with  special  needs,  integration 
of  subjects,  and  transparent  use  of  technology. 

In  analyzing  instructional  methods,  all  respondents  identified  that  an  expert  would  have  spent  more  time  in 
front  -end  planning  and  preparation  such  as  more  detailed  initial  learner  analysis  of  their  backgrounds,  values,  skills, 
and  learning  styles. 

If  you  have  students  who  do  not  do  well  with  independent  activities  and  research,  then  they  will 
not  do  well  with  this  project. 

[Becky ] should  have  incorporated  background  knowledge  into  the  lesson 

a)  needed  information  on  how  to  do  the  assigned  tasks  (research,  writing,  editing,  organizing) 

b)  she  should  have  taught  the  students  how  to  properly  use  the  technology  (and  she  should  have  learned 
too) 

c)  she  should  have  had  a lesson  on  what  is  appropriate  to  video  tape  and  what  is  not 

The  respondents  were  clearly  aware  of  the  protagonist’s  classroom  management  difficulties.  They 
recognized  that  Becky  was  easily  distracted  in  the  classroom.  They  also  observed  that  she  liked  to  teach 
and  wanted  to  make  the  classroom  a good  experience  for  her  students. 

Becky  did  not  have  much  control  over  what  she  was  teaching.  She  did  not  start  the  whole 
introduction  well.  She  needed  to  set  guidelines  and  goals  for  the  students  to  reach.  They  needed  to 
know  exactly  what  she  expected  from  them.  She  insisted  that  it  would  be  a fun  project.  Each  time 
she  introduced  something  new  she  should  have  set  up  their  rules  and  reasons  why  they  were  doing 
that  portion  for  the  project.  She  needed  to  be  clear  that  it  was  not  just  a film-making  project. 

Not  surprisingly,  all  respondents  identified  Becky  Dawson’s  shortcomings  in  addressing  the  special  needs 
of  one  of  her  students.  All  respondents  addressed  the  role  of  the  Individualized  Education  Plan  (lEP)  within  the 
case.  One  response  implied  the  teacher  had  not  consulted  the  lEP.  Others  went  into  more  depth;  exploring  the 
compound  inter-relationship  among  failure  to  assign  tasks  that  were  both  specific  and  appropriate  for  the  student,  the 
focus  on  a primary  technology  at  the  expense  of  others,  and  the  classroom  management  issues  that  culminated  in 
letting  a quiet,  special  needs  student  slip  through  the  cracks. 

For  students  with  special  needs,  it  is  very  important  to  know  exactly  what  accommodations  and 
modifications  need  to  be  made  before  the  lessons  even  begin.  You  need  to  actually  decide  on  how  to  teach 
the  lesson  and  what  you  expect  them  to  learn.  Instead  of  having  the  special  needs  students  do  the  research 
and  screen  writing.  She  may  have  been  able  to  make  a poster  aboard  or  a different  type  of  project  hat  will 
incorporate  her  goals  and  class  objectives.  We  need  to  know  what  her  needs  are  what  she  expected  to 
know.  You  also  have  to  know  if  she  is  able  to  be  video  taped. 
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She  needed  to  set  up  specific  tasks  for  each  group  member.  She  needed  to  set  up  specific  tasks  to  meet  lEP 
goals  for  special  education  students.  She  needed  a rubric  for  the  project  so  students  would  know  what  was 
expected  of  them  and  so  they  could  keep  on  task 

All  four  respondents  identified  the  protagonist’s  failure  to  integrate  subjects  effectively.  Three  of  four 
respondents  commented  about  integrated  subjects  within  the  contact  of  Becky’s  political  naivete  with  her  fellow 
teachers.  Although  we  wanted  the  students  to  identify  and  relate  to  the  political  ramification  that  occur  within  a 
school  setting,  we  had  anticipated  that  this  response  would  be  oriented  to  the  sixth  grade  project.  No  team,  however, 
noted  the  positive  support  from  the  principal.  This  response  would  suggest  an  opportunity  for  improving  the  case. 
Becky  was  a fairly  new  teacher  with  little  experience  in  implementing  a thematic  unit  using  technology. 

She  should  have  talked  with  other  teachers  to  incorporate  other  subject  areas  and  the  teachers  into  the  project.  She 
could  have  the  technology  club  come  into  the  classroom  and  tape  the  projects 

Only  one  respondent  addressed  transparent  use  of  technology  as  an  isolated  item.  Further  investigation  is 
needed  on  this  point.  Perhaps  they  felt  they  had  discussed  technology  concerns  adequately  elsewhere  in  their 
analysis.  It  might  indicate  as  well  the  phrase  “invisible”  or  “transparent”  was  not  meaningful  to  this  particular 
population. 

The  technology  could  have  been  used  instead  to  gather  information  or  used  [in ] other  multimedia 
presentations  that  were  not  so  focused  on  making  a film. 

Actions : Next,  the  respondents  posted  potential  solutions  to  their  identified  issues,  describing  what  a 
novice  teacher  might  do  in  the  same  situation  and  what  challenges  he  or  she  might  encounter.  All  respondents 
identified  classroom  management  as  critical  problem  area.  They  recommended  specific  task  assignments  and  in 
some  cases  alternate  assignments. 

She  needed  to  set  up  specific  tasks  for  each  group  member. 

She  needed  to  have  some  “down  time”  to  have  the  students  adjust  and  get  ready  for  their  next 
class 

The  respondents  specified  numerous  actions  that  Becky  might  consider  for  scheduling.  Some  suggestions 
interfaced  with  the  suggestions  for  enhancing  classroom  management  such  as  due  dates  for  individuals  tasks.  They 
also  recommended  that  Becky  should  have  scheduled  more  time  to  prepare  herself  for  the  project.  The  respondents 
recognized  that  at  a novice  teachers  can  be  overwhelmed  with  the  sheer  range  and  volume  of  activities  in  a 
classroom,  and  that  help  she  should  have  reasonably  expected  wasn’t  available.  Nevertheless,  they  felt  many  of  the 
scheduling  problems  might  have  been  avoided  with  greater  attention  to  detail. 

[Becky]  needed  to  have  a qualified  assistant  to  help  with  the  video  equipment  and  should  have  talked  about 
scheduling  concerns. 

She  should  have  scheduled  conferences  to  see  student  progress 

When  analyzing  student  use  of  video  technology,  the  respondents  readily  acknowledged  the 
instructional  gap  between  attending  a technology  workshop  and  implementing  that  technology  into  a major 
instructional  unit.  They  also  recognized  that  a number  of  ethical  and  behavioral  issues  are  associated  with 
technology  usage. 

She  should  have  tried  to  use  the  video  equipment  herself  to  see  if  it  worked 

She  shouldjhave]  scheduled  a lesson  to  teach  students  how  to  use  the  equipment  properly  and  with 
the  right  amount  of  respect 

Case  participants  missed  some  aspects  of  the  question  regarding  student  use  of  software  for  design  and 
promotion,  mostly  answering  in  the  context  of  rubrics  for  grading.  Elements  of  desktop  publishing  software  and 
other  applications  the  middle  school  students  in  the  case  might  have  used  had  been  edited  from  the  case.  However, 
they  had  already  addressed  video  related  concerns  earlier.  This  was  not  an  effective  question  and  may  be  revised  or 
removed  be  in  future. 

She  could  have  used  a rubric  to  explain  what  was  required  for  the  design  and  promotion 

The  respondents  identified  numerous  tactical  improvements  in  the  political  arena.  Participants  recognized 
that  the  principal  was  supportive  of  technology  integration  and  wished  for  Becky’s  success.  All  were  quite  clear 
about  describing  problems  that  stemmed  from  Becky’s  lack  of  buy-in  from  here  colleagues  in  the  idle  school. 

She  should  have  presented  the  project  to  the  administration  to  see  what  they  would  think  about  it  she 
should  have  involved  her  colleagues  more  with  the  preplanning,  planning,  and  post-planning.  She  should 
discuss  their  concerns  over  the  project.  She  should  have  asked  for  ideas  on  how  to  integrate  their  subjects 
into  the  project 
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All  respondents  suggested  means  for  incorporating  standards  into  the  project.  Some  suggestions  included 
communication  with  parents  to  inform  them  of  the  impending  project  and  generate  interest,  increased  coordination 
with  her  administration,  and  development  of  lesson  plans  that  correlated  objectives  to  state  standards. 

She  should  also  send  a letter  home  to  the  parents  letting  them  know  about  the  project  ahead  of  time.  Also  it 
would  be  a good  idea  for  Becky  to  run  the  project  past  her  superiors  to  get  their  input/approval  including  a 
detail  description  of  how  this  project  will  meet  the  state  standards. 

The  most  important  goal  of  creating  projects  is  how  it  meets  standards  and  standardizes  testing 
requirements.  She  should  have  had  these  clear  before  making  out  the  goals  and  objectives  for  her 
lessons.  She  could  then  show  others  how  her  project  does  fit  into  the  curriculum  and  testing 
requirements. 

Consequences : The  respondents  next  considered  and  described  the  possible  consequences  of  their 
proposed  actions.  One  respondent  was  clearly  on  track,  but  brief.  Two  of  the  four  respondent  comments  were  very 
detailed  and  carefully  considered.  The  remaining  respondent  integrated  actions  and  consequences  into  a single 
narrative  stream.  All  comments  were  relevant  and  pertinent.  Overall,  respondents’  comments  reflected  the 
potentially  beneficial  consequences  and  were  well  delineated;  the  possible  negative  outcomes  were  less  well 
represented  in  the  discussion  forum.  Perhaps  the  participants  figured  the  protagonist’s  situation  couldn’t  get  much 
worse.  As  one  team  member  observed,  “I  wouldn't  want  to  be  in  Becky’s  shoes”. 

By  hearing  her  colleagues’  concerns  prior  to  implementing  the  project,  Becky  may  have  to  further 
delay  starting  the  project  in  order  to  meet  with  all  of  her  colleagues.  Meeting  with  the  special 
educators  may  also  delay  the  project.  Sending  a letter  home,  again  will  take  time  away  from  the 
implementation  of  the  project.  Also  there  may  be  more  questions  and  concerns  for  Becky  to  deal 
with  requiring  more  of  her  time  that  she  is  already  lacking.  By  getting  the  approval  of  her 
superiors,  Becky  might  run  into  financial  concerns  from  the  administration.  Also  this  approval 
process  [may]  take  time  away  from  the  implementation  of  the  project.  The  students’  contracts 
may  result  with  a student  losing  the  privilege  of  using  the  technology,  thus  losing  the  point  of  the 
project. 

Recommendations : Respondent  recommendations  for  corrective  action  for  Becky  were  similar  to  one 
another.  Two  respondents  recommended  that  the  protagonist  try  the  project  again,  perhaps  in  the  next  year.  They 
suggested  that  a period  of  careful  assessment  and  implementation  of  specifically  improvements  detailed  in  their 
earlier  case  analysis.  Their  suggested  targets  included  improvements  in  unit  plans,  lesson  plans,  consultations  with 
special  needs  students,  becoming  more  familiar  with  the  technology  and  some  serious  networking  with  her 
colleagues  and  administrators.  One  respondent  recommended  that  Becky  “assess  the  students  on  what  they  have 
learned  and  set  new  goals’,  making  corrective  action  immediately.  The  fourth  respondent  refined  an  earlier  list  of 
actions,  but  suggested  no  time  frame. 

?Becky  should  assess  the  students  on  what  they  have  learned  and  set  new  goals  on  what  they  have 
not  learned  that  will  be  needed  for  the  standardized  assessment.  She  should  explain  that  they  need 
to  focus  on  these  skills  and  they  are  responsible  to  know  these  aspects  for  the  end  of  the  marking 
periods.  She  then  needs  to  find  an  alternative  to  the  filmmaking.  Her  ideas  were  exciting,  just  not 
necessarily  appropriate  at  this  time.  She  could  still  have  the  students  research  topics  and  share 
with  others  but  they  need  to  be  meaningful  to  the  class  standards. 

Other:  All  teams  wrote  that  this  case  would  help  them  to  transition  from  novice  to  expert  regarding  the 
classroom  integration  of  technology.  All  respondents  noted  that  case  scenario  increased  their  awareness  of  the 
importance  of  careful  planning  and  implementation  in  order  to  optimize  successful  integration  of  technology  into 
their  own  classrooms.  One  respondent  added  that  more  consideration  needs  to  be  taken  for  special  education 
students  and  their  use  of  and  difficulties  with  technology  in  the  classroom. 

By  reading  this  case  scenario,  we  are  better  able  to  take  into  consideration  the  problems  that  may  arise 
from  implementing  technology  into  our  own  classrooms.  We  now  realize  that  this  implementation  needs  to 
be  done  slowly  with  a great  amount  of  preparation  in  order  for  it  to  be  successful. 

The  entire  activity,  including  discussion  of  the  case  and  posting  of  responses,  was  completed  in  less  than  or 
equal  to  2.0  hours.  The  participants  felt  this  case  was  easier  than  the  analysis  of  Terry  Kirkland,  the  case  used  for 
training.  A few  commented  that,  as  pre -service  or  in-service  teachers,  they  could  relate  more  closely  to  the  character 
Becky  Dawson  than  to  Terry  Kirkland. 
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Conclusion 

The  case  participants  in  each  team  successfully  identified  the  majority  of  the  primary  issues  intended  by  the 
case  designers.  As  might  be  expected,  the  participants  with  classroom  experience  responded  in  greater  depth  with 
more  specific  suggestions  or  analysis.  Participants  were  actively  engaged  in  discussions  that  preceded  the  posted 
case  response.  All  felt  the  case  would  enhance  their  capacity  to  plan  integration  of  technology  into  their  classrooms. 
Two  major  themes  emerged  from  posted  responses 

• Integrating  technology  for  the  first  time  requires  taking  risks  and  embracing  both  successes  and  failures, 
but  has  some  reward.  They  are  cognizant  that  technology  integration  is  a lengthy  process  that  requires  a lot 
of  investment. 

• The  respondents  demonstrated  their  use  of  reflective  practice  by  describing  their  knowledge  and  prior 
experience  in  learning  and  teaching  and  then  relating  that  to  the  challenges  that  Becky  encountered. 

By  reading  this  case  scenario,  we  are  better  able  to  take  into  consideration  the  problems  that  may 
arise  from  implementing  technology  into  our  own  classrooms.  We  now  realize  that  this 
implementation  needs  to  be  done  slowly  with  a great  amount  of  preparation  in  order  for  it  to  be 
successful. 

Discussion  and  Future  Development 

Some  possible  avenues  for  research  and  further  exploration  of  the  case  include  the  following: 

• As  teacher  education  and  P-12  institutions  guide  novice  teachers  in  their  development  of  the  skills  and 
abilities  needed  for  successful  practice,  case  studies  can  provide  experiences  to  address  questions  such  as: 
What  concepts  and  methods  enhance  this  learning  experience  and  enable  novice  teachers  to  transfer  what 
they  learned  to  actual  practice?  How  might  case  studies  help  to  address  this  question? 

• The  role  of  disposition  is  not  addressed  in  this  cas  e.  This  is  a case  that  could  easily  be  revised  to  address 
this  issue  among  others  and  will  be  considered  by  the  authors. 

• This  case  is  well  suited  for  adaptation  to  a multimedia  format  with  video  clips,  additional  ancillary 
supplements  such  as  the  Jefferson  Middle  School  Technology  Plan,  lEP,  and  interim  student  work  samples. 

• The  case  should  be  tested  with  a larger,  representative  group  of  in-service  teachers. 
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Reflections  on  Educational  Technology  and  System  Science 
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Abstract 

The  contribution  of  systems  science  is  of  critical  importance  as  a foundational  paradigm  and  theory,  which 
has  extensive  influence  on  the  growth  of  theories  and  practices  of  educational  technology.  Unlike  system  science 
that  evolves  and  develops  through  convergence  and  divergence,  however,  educational  technology  does  not  reflect  on 
these  changes  even  though  it  is  based  on  system  science.  This  paper  reviews  the  main  concepts  and  theories  of 
systems  science  as  the  basis  theory  of  educational  technology  and  explores  new  methods  for  the  paradigm  and 
methodology  of  systems  science  to  support  educational  technology  by  analyzing  and  linking  educational  technology 
and  system  science.  One  of  the  major  findings  of  systems  science  is  that  educational  technology  has  a variety  of 
educational  situations  and  systems,  and  each  system  is  unique.  In  a nutshell,  one  alternative  direction  that 
educational  technology  suggests  based  on  this  finding  is  that  although  it  has  provided  solutions  to  resolve  current 
problems,  it  should  develop  to  the  point  where  it  is  able  to  construct  preventative  measures  in  the  future. 

Introduction 

Systems  science  has  served  as  a foundation  for  the  development  and  growth  of  educational  technology  as 
its  foundational  theory  and  paradigm.  Systems  science,  the  parent  theory  has  undergone  various  changes  and 
developments  with  criticism  and  self-re  flection;  however,  educational  technology  has  been  criticized  for  its  lack  of 
growth  and  self-reflection.  It  is  high  time  to  review  the  existing  paradigms,  theories,  and  methodologies  that 
underpin  the  very  foundation  of  system  science,  and  this  should  be  linked  to  voluntary  changes  within  educational 
technology.  This  paper  explores  methods  for  systems  science  to  effectively  support  the  growth  of  educational 
technology  by  1)  reviewing  special  qualities  of  systems  science  as  a basis  theory  of  educational  technology,  7) 
examining  major  theories  and  concepts  of  systems  science,  and  3)  analyzing  the  current  educational  technology  in 
link  with  systems  science  paradigm. 

Review  of  Systems  Paradigm  of  Educational  Technology 

Systems  science  has  been  applied  to  educational  technology  from  1950  and  1960  to  the  present  (Saettler, 
1990;  Finn,  1956;  Finn,  1965;  Seels  & Richey,  1994).  Great  increases  in  language  labs,  programmed  instmction, 
simultaneous  use  of  various  media,  and  instructional  computers  in  the  ’50s  and  ’60s  gave  birth  to  systems 
approaches  that  focus  on  integrated  systems  and  processes.  In  the  1960s,  the  total  systems  approach  was  coined  to 
explain  interaction  between  human  beings  and  machinery  in  organization  theory  (Saettler,  1990). 

In  the  1970s,  the  Association  for  Educational  Communications  & Technology  (AECT)  defined  educational 
technology  as  a systematic  method  that  designs,  executes  and  evaluates  the  whole  process  of  learning  and  instruction 
according  to  specific  learning  gpals  based  on  human  learning  and  communication  theories  (Commission  on 
Instructional  Technology,  1970,  P.  21).  Definitions  of  Silber  (1970),  MacKenzie  and  Eraut  (1971)  and  AECT 
(1972)  value  the  word,  systematic,  as  a major  concept  of  educational  technology  and  it  is  the  most  common  word  in 
their  definitions.  Ely  (1973)  also  pointed  out  that  a systematic  approach,  a study  of  means,  and  a field  directed 
approach  toward  some  purpose  are  the  three  core  concepts  in  the  various  definitions  of  educational  technology.  It  is 
clear  that  early  educational  technology  with  early  system  approach’s  adaptation  was  in  line  with  absolute  and 
recursive  characteristics  that  pursue  structural,  analytical  and  stage  approaches. 

Eor  a few  decades  after  the  1970s,  the  term,  systems  approach  has  been  used  to  explain  procedures 
reflecting  goals  and  means  perspectives  with  heavy  links  to  linear  and  behavioral  concepts  in  instructional  systems 
design  and  development  and  educational  planning.  As  a result,  numerous  people  in  the  educational  technology 
circle  adopted  the  term  as  a systematic,  anti-creative  and  mechanical  approach  that  enables  easier  management  of 
projects  (Romiszowski,  1996).  This  systems  approach  is  a practical  methodology  with  strong  systematic  and  phase 
characteristics,  and  it  has  served  as  a concrete  model  for  integration  of  instructional  system  design.  The  most 
popular  model  is  that  of  Dick  and  Carey. 

Some  systems  scientists,  such  as  Banathy  advocated  systemic  perspective,  systemic  philosophy,  systemic 
research  and  systemic  methodology  by  suggesting  an  instructional  systems  design  that  embodies  expansionism  and 
active,  non-linear,  synthesis -orientation  and  holistic  mode  thinking,  rather  than  classic  scientific  thinking  of  an 
analytic,  restorative  and  linear  causality  approach.  However,  practical  philosophers  in  particular  within  the 
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educational  technology  circle  have  considered  this  model  as  an  excessively  complicated  model  without  practicality. 
As  a result,  this  model  has  not  made  it  into  the  main  streamof  educational  technology. 

According  to  the  definition  and  area  of  educational  technology,  recently  recognized  by  AECT,  the  design 
area  of  educational  technology  evolved  from  instructional  psychology  into  the  instructional  systems  design  area 
through  the  adaptation  of  systems  theory.  Designing  in  this  process  is  defined  as  follow.  Design  embodies  both 
systemic  approach  (macro  level)  and  each  step  within  it  (micro  level).  ISD  is  a typically  linear  and  reoccurring 
process  that  demands  consistency  and  completeness  in  every  stage.  In  ISD,  process  is  as  important  as  outcome  since 
the  validity  of  the  outcome  depends  on  the  process  (Seels  & Richey,  1994).  Even  the  latest  definitions  reiterate  the 
importance  of  process,  stage,  linear  aspect,  recurrence,  completeness,  consistency,  and  balance,  signaling  a shift  to 
analytical,  closed  and  determinist  methodology  from  the  general  systems  theory  of  wholeness,  dynamic  interaction, 
self-control  and  openness . 

Saettler  (1990)  points  out  that  Gagne  and  Briggs’s  instructional  systems  models  and  other  models  are  not  or 
merely  related  to  general  systems  theory  but  the  assumptions  and  provided  processes  are  more  related  to 
instructional  design  and  system  analysis  than  to  instructional  ‘systems’  design.  Yet,  educational  technologists  and 
curriculum  designers  are  trying  to  rationalize  their  program  designing  approaches  with  general  systems  theories, 
according  to  Saettler.  Yoo  (1994)  raised  a question  about  whether  the  current  systems  approach  regarding  ISD  can 
be  viewed  as  a real  systems  approach  in  its  essence. 

System  Science 

Systems  inquiry  is  valued  as  a study  that  introduced  new  thinking,  overhauling  the  classic  science 
paradigms  of  analytical  thinking,  reductionism,  and  determinism.  Systems  paradigms  embody  synthetic  thinking, 
emergence,  communication  and  control,  expansionism,  and  teleology.  Eirst,  synthetic  thinking  conceptualizes  the 
object  (system)  of  understanding  as  a part  of  a bigger  whole  comprised  with  one  or  more  parts,  not  as  a whole  that 
should  be  divided  to  be  understood.  Second,  another  new  concept  brought  by  systems  inquiry  is  expansionism  as  an 
alternative  for  reductionism,  and  this  suggests  that  ultimate  understanding  is  not  something  palpable  but  an  ideal 
stage  that  learners  can  get  continuously  closer  to.  Continuous  progress  toward  ultimate  understanding  is  decided  by 
the  scope  of  understanding  on  a bigger  and  more  comprehensive  whole.  Third,  the  idea  of  non-deterministic 
causality  (Singer,  1959)  developed  subjective  teleology  - a conceptual  system  in  which  teleological  concepts,  such 
as  free  will,  free  choice,  free  function  and  free  intention  can  be  functionally  defined  and  integrated  (Cavallo,  1979). 

General  Systems  Theory 

Studying  of  paradigms  and  core  concepts  of  GST  makes  it  clear  that  numerous  traditional  instructional 
system  models,  including  models  of  Gagne  and  Briggs  and  Dick  & Carey,  are  not  or  merely  related  to  GST  but  only 
part  of  theories,  such  as  wholeness,  negative  feedback  and  boundary  concepts  are  related  to  GST  to  some  extent. 

Biologist  Ludwig  Von  Bertalanffy  realized  the  limitation  of  reductionistic  scientism  and  introduced  GST  as 
an  inter-discipline  focused  on  wholeness  and  dynamic  interactions.  He  is  one  of  the  scientists  who  paid  early 
attention  to  the  difference  between  closed  systems  and  open  systems.  Differentiation  of  open/closed  systems  has 
served  as  a foundation  for  systemic  thinking  and  has  been  helpful  for  the  paradigm  shift  away  from  the  absolutism 
of  Newtonian  science.  Since  this  development,  GST  has  served  as  a foundation  for  various  system  theories  and 
methodologies  and  it  has  evolved  and  expanded  into  various  philosophies,  theories  and  methodologies  in  numerous 
academic  areas.  Soft  systems  theory  with  strong  characteristics  of  methodology  developed  basic  philosophy  and 
theories  of  GST  into  a concrete  solution  for  problems  of  HASs.  GST  gave  birth  to  the  following  concepts  that 
define  the  essential  characteristics  of  systems. 

Wholeness:  Systems  developed  by  organic  creatures  and  human  beings  are  unified  wholes  with  integrated 
and  sustaining  rules  and  regulations.  Properties  of  a system  are  not  a simple  sum  of  subsystems  or  sub-elements  but 
those  of  the  whole  system. 

Openness  and  Boundary:  Organisms  including  the  universe,  mankind  and  living  creatures,  or  systems 
made  by  human  beings  are  not  things  that  exist  in  a closed  universe.  They  can  exist  without  falling  apart  into  pieces, 
since  they  have  the  ability  to  exchange  information,  energy,  and  substances  with  their  environment  through 
permeable  boundaries. 

Dynamics:  System  function  has  a generic  characteristic  of  emergence,  since  each  part  of  a system  interacts 
with  at  least  one  of  others.  Changes  within  an  organization  depend  on  interactions  and  structural  relations  of  sub- 
groups, not  on  individuals.  It  is  also  a characteristic  of  GST  that  conceptualizes  systems  dynamics  in  relation  to 
inter-connectivity . 

Feedback : The  prerequisite  for  system  feedback  is  that  there  are  two  different  types  of  feedback:  negative 
and  positive.  Negative  feedback  corrects  systems  to  return  within  the  radius  accepted  by  a controlling  mechanism 
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when  the  system  deviates  from  self-determined  or  programmed  objectives  or  status.  Negative  feedback  encourages 
more  stable,  adaptive  and  goal-oriented  systems.  Positive  feedback  guides  the  system  to  the  changing  direction  of 
output  (the  current  status),  thereby  maintaining  growth  or  reduction. 

Interconnectivity:  The  perspective  that  everything  is  inter-connected  suggests  every  system  is  not  isolated 
but  inter-dependent  and  inter-connected  and  the  causalities  are  often  not  linear.  More  extreme  GST  argues  that  the 
system  evolves  as  a result  of  interaction  with  its  environment  and  co-evolves  with  the  environment  due  to  its  generic 
characteristic  of  interconnectivity.  It  means  that  the  environment  changes  due  to  interaction  with  its  sub-elements. 
Banathy  (1991),  Gareth  Morgan  (1986)  and  Birch  (1990)  are  some  of  those  who  advocated  co-evolution  of  systems. 
Equifmality:  Equifinality  is  a system’s  ability  that  ensures  systems  to  reach  the  same  end,  even  with  different 
starting  points  and  different  methods  (Bertalanffy,  1956).  Equifinality  implies  that  systems  are  moving  toward  a 
common  goal  and  it  is  adoptive  since  it  has  self-regulation  and  autopoiesis;  moreover,  it  has  significant  meaning  in 
the  management  of  human  beings  and  organizations.  Equifinality  also  suggests  that  there  cannot  be  one  single 
correct  way  but  there  can  be  various  right  ways,  depending  on  the  changing  conditions  of  the  environment. 
Therefore,  prescribing  a single  strategy  or  method  to  achieve  specific  goals  and  excessive  adherence  to  that  method 
will  not  be  efficient. 

Soft  Systems  Theory 

Unlike  hard  systems  theory  with  systems  engineering  and  operations  research  for  resolution  of  technical 
issues,  soft  systems  theory  views  human  systems  and  social  systems  as  a soft  system,  and  it  has  gained  more 
popularity  since  the  1970s.  In  particular,  the  distinction  between  hard  systems  and  soft  systems  seems  to  be  related 
to  the  work  of  Checkland  and  Churchman.  These  two  researchers  examine  theories  and  approaches  within  the 
framework  of  soft  systems  concepts  for  dealing  with  human  activity  systems  (HASs)  in  which  any  output  cannot  be 
predicted,  and  factors  outside  the  designer’s  control  can  emerge  in  the  inner  system. 

Soft  Systems  Methodology  (SSM):  Checkland  argues  that  natural  science  methodologies  cannot  be  applied  to  ill- 
structured  issues  and  further  suggested  a systems  methodology  called  CATWOE  to  study  soft  systems.  As  it  is  well 
proclaimed  in  this  model,  basic  conditions  of  soft  systems  theory  are  sustained  and  iterative  methodologies  and 
directions  of  changes  and  execution  are  set  by  stakeholders’  views  and  values  through  dialogues  and  discussions. 
Systems  Design:  Banathy  is  one  of  the  strong  advocates,  who  view  soft  systems  theory  and  its  methodology  as  the 
most  ideal  educational  planning  and  instmctional  system  design.  Emerging  design  inquiry  seems  to  be  dynamic  and 
wholistic;  iterative  and  integrative;  spiral  and  interactive;  participative,  dialogue-oriented  and  consensual  (Banathy, 
1987,  P95).  His  system  methodology  is  a spiral  design  with  a use-designer  approach  (Banathy,  1987;  Banathy, 
1991;  Lee,  1995). 

The  concept  of  ‘user  = designer’  is  based  on  the  belief  that  systems  can  be  most  successful,  executable  and 
productive  and  the  commitment  for  execution  is  the  most  binding  force  if  it  is  managed  by  future  users  of  the 
system.  Banathy  (1998)  suggested  that  HASs,  such  as  family,  learning  systems,  instructional  systems  and  education 
systems  have  five  different  types  of  systems  based  on  (1)  closeness  and  openness,  (2)  simplicity  and  complexity,  (3) 
mechanical  and  systemic  characteristics,  and  (4)  unitary  and  pluralistic  objectives.  Combination  of  these  four 
dimensions  can  lead  to  five  different  types  of  systems:  (1)  rigidly  controlled  systems,  (2)  deterministic  systems,  (3) 
purposive  systems,  (4)  heuristic  systems  and  (5)  purpose  seeking  systems  (Banathy,  1991).  Eirst,  rigidly  controlled 
systems  are  closed  with  restricted  and  well-controlled  interaction  with  its  environment.  Second,  deterministic 
systems  are  unitary  since  they  are  open  to  their  environment  but  still  have  stronger  characteristics  of  closeness  and 
clearly  defined  objectives.  Third,  purposive  systems  are  still  unitary  with  defined  objectives,  but  they  have  freedom 
of  choice  for  functional  objectives  and  methods.  Eourth,  heuristic  systems  are  more  or  less  pluralistic  since  they 
decide  their  own  goals  under  the  general  guideline.  Lastly,  purpose-seeking  systems  are  ideal-seeking,  directed 
according  to  their  visions  of  the  future. 

Cybernetic  Theory 

Cybernetics  is  a stream  of  system  theory  and  has  evolved  into  first-order  cybernetics  and  second-order 
cybernetics.  Cybernetics  has  the  closest  connection  with  educational  technology  among  various  system  theories 
(Winn,  1975).  It  is  a mechanism  of  early  cybernetics  system  control  with  a feedback  concept  and  second-order 
cybernetics;  in  particular,  it  provides  strong  supporting  concepts  on  constructive  position  (Bopry,  1999).  It  is  shown 
in  Table  1 that  the  two  streams  of  cybernetics  have  different  but  complimentary  perspectives. 

First-order  Cybernetics:  Eirst-order  cybernetics  is  a perspective  that  harnesses  information  on  output  with 
feedback  as  a controlling  mechanism.  This  perspective  quantifies  information  that  moves  from  one  place  to  another 
and  ignores  the  contents,  meaning,  and  intention  of  the  information  (Gleick,  1987).  Eirst-order  cybernetics  deals 
with  systems  issues  through  technical  rationality  based  on  traditional  objectivism  and  absolutism  Eirst-order 
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cybernetists  stand  in  a position  to  observe  phenomenon  and  what  they  describe  about  the  phenomenon  is  their 
experiences  with  the  phenomenon.  As  a result,  systems  are  objects  to  be  observed.  This  perspective  is  interested  in 
internal  communfcation  and  control,  and  it  focuses  on  a causal  feedback  mechanism  with  cyclical  process  of  self- 
organization and  self-regulation.  This  principle  has  laid  the  foundation  for  computer/cognitive  science  and  it  is 
viewed  as  the  neural  network  approach  in  comp  uter  science. 

Second-order  Cybernetics:  Foerster  (1984)  coined  the  word  second-order  cybernetics.  First-order 

cybernetics  focuses  on  feedback  while  second-order  cybernetics  on  reflexivity;  therefore,  the  latter  tries  to 
understand  and  solve  phenomena  through  their  relations  with  other  systems.  Second-order  cybernetists  include 
themselves  as  a part  of  phenomena  and  systems  to  be  observed  (Foerster,  1972).  Therefore,  this  perspective  leads  us 
to  consider  that  context,  contents,  meaning  and  intention  of  faformation  are  the  center  of  the  observance,  thus 
reiterating  the  importance  of  language,  culture  and  communication  as  analysis  methods  in  order  to  observe  and 
describe  the  phenomenon.  Second-order  cybernetics  tries  to  inquire  about  the  meaning  of  cognition  and 
communication  in  social  implementation  studies,  such  as  natural  science,  social  science,  humanity,  information 
science,  construction,  education,  organization,  art,  management  and  politics  through  the  concept  of  self-reference. 


Table  7.  Complimentary  Perspectives  of  First-order  and  Second-order  Cybernetics  (Bai,  1999) 


Focus 

First-order  cybernetics 

Second-order  cybernetics 

Feedback  types 

Negative  feedback 

Positive  feedback  & negative  feedback,  feedforward 

Pursuing  status 

Stability,  homogeneity 

Morphogenesis,  development,  self-adaptation, 
autopoiesis 

Objects  of  interests 

Mechanic  system 

Biological  system,  organic  system,  social  systems 

Instruments 

Feedback  control 

Self -references,  cooperation,  learning,  communication 

Goal-creator 

External  system 

Internal  system 

Views  on  reality 

Independent  from  observers 

Interaction  between  observers  and  the  observed 

Autopoiesis  Theory:  Autopoiesis  theory,  coined  by  Maturana  and  Varela,  is  a type  of  second-order 
cybernetics.  It  is  a complex  and  delicate  theory,  which  explains  language,  human  behaviors,  cognition,  self- 
consciousness  and  living  systems  as  the  entity  experiencing  continuous  autopoiesis  process  (Luhmann,  1995). 
Autopoiesis  systems  are  autonomic  since  they  do  not  depend  on  an  external  production  process  and  have  a 
characteristic  of  self -reference  due  to  self-defined  structure  (Mingers,  1997).  Autopoiesis  theory  explains  the 
importance  of  relations,  meaning  and  communication  of  systems  through  the  concept  of  structural  coupling. 

Structural  Coupling:  Structural  coupling  occurs  when  two  systems  or  one  system  and  its  environment  co- 
exist with  structural  interaction  and  experience  iterative  interaction  within  the  environment  while  maintaining  their 
identities  (Mingers,  1997).  All  social  systems  have  structural  coupling  among  their  organisms  with  neural  systems 
that  fine-tune  behaviors  of  other  systems  and  human  social  systems,  including  family.  Within  these  systems,  regular 
patterns  of  behavior  are  manifested  through  structural  coupling  of  members  (Mingers,  1997),  and  the  formations  of 
harmonious  relations  and  consensual  domain  are  possible  through  communication  through  language  or  linguistic 
behaviors. 

Meaning  and  Communication:  When  social  systems  are  viewed  as  a structural  pattern  of  behaviors 
(Luhmann,  1995),  a social  system  has  to  make  choices  to  survive  and  grow  within  its  complex  structure,  and  this 
process  is  affected  by  various  factors.  However,  it  makes  decisions  as  a principle  body.  This  decision-making 
enables  a system  to  construct  an  independent  meaning  itself.  This  social  system,  also  called  as  system  of  sense, 
communicates  with  other  systems  by  assimilating  their  meanings  (that  is,  sharing  the  method  for  reducing  the 
complexity  of  each  sub-element  or  system).  Systems  sometimes  communicate  in  order  to  reinforce  their 
accumulated  sense  or  to  replace  the  existing  meanings  or  to  adapt  themselves  to  the  environment.  This  method  leads 
systems  to  be  more  complicated,  thereby  making  them  more  differentiated  from  their  environments  (Bausch,  1997). 

Self-reflection:  Systems  become  more  flexible  as  their  internal  complexities  get  intense  and  they  better 
understand  the  contingency  of  choices.  As  a result,  systems  start  learning  not  only  about  themselves  but  also  their 
environments  and  use  self-reflection  and  self-examination  capabilities  to  devise  sub -systems  that  are  controlled  by 
preceding  systems  to  deal  with  environment. 
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Implications  of  Systems  Science 

Although  systems  science  has  grown  and  developed  with  an  evolutionary  manner  through  conversion  and 
diversion  since  the  1960s,  educational  technology  has  lacked  reflection  of  these  changes  and  growth.  Reviewing 
concepts  and  terms  of  systems  science  that  have  been  applied  to  educational  technology  raises  a question  about 
whether  system  theory,  particularly  general  system  theory,  has  actually  served  as  the  pillar  of  educational 
technology  and  whether  it  is  fully  understood  or  not.  If  systems  science  is  the  foundation  of  educational  technology, 
the  educational  technology  academic  should  be  more  adoptive  to  the  development  of  system  paradigms  and 
methodologies  and  should  use  that  theoretical  development  as  the  basis  for  practices  of  educational  technology. 
Fortunately,  faced  with  a variety  of  changes,  educational  technology,  especially  the  BD  area,  is  showing  signs  of 
changes  and  self-reflection  (Wilson,  1997;  Wilson,  1999;  Ritchie  and  Earnest,  1999).  It  airs  a voice  of 
reconsideration  to  present  new  directions  at  least  in  three  areas. 

(1)  Excessive  inclination  to  determinism:  traditional  instructional  systems  design  models  widely  accepted 
and  utilized  are  linear,  based  on  behaviorism  and  reflecting  goals -means  approaches.  These  models  are  overly 
simplified,  thus  lack  flexibility  (Saettler,  1990;  Richey,  1995).  Instructional  systems  design  theories  and  their 
models  based  on  rational  and  ‘mechanical  rationales’  (Schon,  1987)  imply  problems  of  not  appropriately  reflecting 
the  design  process  of  professional  instructional  systems  designers.  Experts  are  highly  flexible  and  very  adaptive  in 
application  of  their  knowledge  to  real  problems  (Nelson,  Magliaro  & Sherman,  1988;  nelson  & Orey,  1991; 
Thiagarajan,  1976).  They  do  not  simply  apply  formulated  theories  but  develop  their  own  models  by  discarding 
unnecessary  factors  with  their  own  discretion  and  combining  models  and  ideas  (Tessmer  & Wedman,  1992; 
Wedman  & Tessmer,  1990).  Then,  instructional  systems  design  theories  that  are  utilized  to  resolve  ill-defined 
problems  are  cornered  with  their  prescription  adherence  to  design  process  and  final  outputs 

(2)  Gap  with  context:  traditional  instructional  systems  design  theories  and  models  are  based  on  the 
perspectives  that  learning  becomes  most  efficient  when  a learner  is  trained  away  from  daily  business  and  tasks.  In 
addition,  instructional  systems  design  models  sometimes  do  not  reflect  how  training  is  designed  and  developed  in 
‘real  world’  (Dick,  1995).  Instructional  systems  development  should  be  more  responsive  to  market  demands. 

(3)  Excessive  process  orientation:  learning  principles  and  theories  have  always  been  integrated  into 
traditional  micro  designing  models,  such  as  Gagne’s  learning  events  and  Merrill’s  CDT  and  have  been 
recommended  to  be  grafted  to  instructional  systems  design  methods.  However,  many  experts  have  expressed  their 
concerns  over  the  typical  tendency  of  instructional  designers,  who  stress  the  design  ‘process’,  emphasized  in  models 
but  they  overlook  the  implementation  of  learning  principles  (Winn,  1989). 

Systems  theory  experienced  a weakening  of  recognition  in  the  1980s,  but  fortunately  made  a strong 
rebound  in  the  1990s.  There  are  some  movements  to  adopt  systems  theories  and  systemic  thinking  for  educational 
system  designing  and  educational  reform  as  a macro  solution  to  educational  problems  (Banathy,  1991;  Reigeluth, 
1988,  1992;  Lee,  1995;  Lee,  1995)  and  adopt  systems  theory  as  a methodology  to  support  constructivistic  learning 
environment  systems  design  and  hypermedia  and  cyber  educational  systems  design. 

Romiszowski  (1996)  suggested  the  following  important  roles  of  systemic  approach  in  the  above  areas. 
Systemic  perspectives  of  cybernetics  make  it  possible  to  understand  the  complexity  of  real  problems  related  to  all 
the  endeavors  of  restructuring  and  innovation  and  to  present  clear  visions.  Meanwhile,  from  the  perspective  of 
constructivism,  instructional  systems  design  has  shifted  its  focus  to  knowledge-base  creation  that  can  be  commonly 
used  under  the  learner’s  control  from  teacher’s  control  due  to  the  multiplicity  of  learners’  intentions  and  capability. 
Its  focus  shift  to  systemic  analysis  of  knowledge  structure  also  should  be  regarded.  In  particular,  hypertext -based 
knowledge  creation  cannot  choose  but  to  depend  on  highly  sophisticated  systemic  thinking. 

Systemic  Characteristics  of  Multimedia  and  Cyberspace 

There  are  increasing  demands  for  educational  technology  to  tackle  ill -defined  problems  and  highly  open 
systems,  such  as  design,  operation  and  evaluation  of  hypermedia  and  cyber  educational  systems.  As  a result, 
analyzing  system  types  and  applying  appropriate  interventions  and  methodologies  expected  to  be  applied  in  our 
field. 

Multimedia  applications  provide  users  with  endless  path  and  variations  and  massive  information. 
Compared  with  the  web,  however,  they  are  relatively  closed  systems.  In  other  words,  their  scope  for  storage 
capacity  is  limited;  thus,  multimedia  applications  contain  what  des  igners  provide.  As  a result,  learners  have  freedom 
to  move  around  within  the  multimedia  environment,  but  their  choices  are  something  that  is  pre -decided  by  the 
designers.  Unlike  multimedia,  applications  in  cyberspace  are  not  stand-alone  but  open  systems  that  can  be 
indefinitely  linked  according  to  multiple  intentions  and  situations  of  users.  In  general,  designers  cannot  make  users 
take  specific  paths  to  materials  in  cyberspace  that  they  designed  and  provided  and  users  can  have  direct  access  to 
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any  page  in  cyberspace.  In  addition,  designers  cannot  predict  how  future  users  will  make  use  of  these  materials  and 
if  the  materials  are  open  worldwide,  and  designers  cannot  know  which  language  users  will  utilize  them. 

Information  and  structures  are  easily  changed  in  cyberspace.  In  the  case  of  the  web,  browser  types, 
versions  and  configurations  can  generate  web  pages  with  very  different  appearances  and  contents  of  the  web  are 
potentially  indefinite  (Jones  & Farquhar,  1997).  Jones  and  Farquhar  (1997)  understand  that  the  openness  of  the  web 
significantly  affects  design  principles,  thus  making  design  extremely  difficult.  Unlike  the  design  of  traditional 
closed  systems  - for  example,  traditional  instructional  design  or  stand-along  applications  design  - the  web  inevitably 
relinquishes  a significant  part  of  controlling  authority  to  users.  They  classified  the  loss  of  control  that  used  to  be  in 
the  hands  of  instructional  designers  in  terms  of  technical  control  and  curriculum  control. 

According  to  the  system  types  suggested  by  Banathy,  cyberspace  has  the  nature  of  most  heuristic  and 
purpose  seeking  systems.  It  is  a complex  and  pluralistic  system  with  the  characteristics  of  wholeness,  openness, 
dynamics,  interconnectivity,  equifmality,  positive  feedback,  feedforward,  morphogenesis,  development,  self- 
adaptation, autopoiesis,  cooperation,  learning  and  communication.  Elements  and  members  within  this  system  build 
their  own  meanings  by  exchanging  information  through  language,  thus  operate  as  an  autonomic  entity  in  creating 
knowledge. 

Highly  complex  and  open  structures  of  multimedia,  particularly  cyberspace  with  characteristics  of  network, 
overhaul  traditional  learning  and  instructional  theories  of  absolutism.  This  is  because  the  characteristics  of 
relativity,  transformation,  connectivity,  interaction,  uniqueness  and  non-formality,  and  digital  are  overwhelmingly 
strong  to  be  understood  and  supported  by  traditional  paradigms  and  theories.  Therefore,  it  requires  alternative 
methodologies  and  alternative  paradigms  and  theories  that  can  properly  interpret  these  phenomena.  As 
Romiszowski  (1996)  suggested,  the  creation  of  a hypertext  knowledge-base  depends  on  a high  level  of  systemic 
thinking  to  analyze  structures  of  specific  knowledge  domains,  to  design  optimal  methods  representing  this  structure 
within  the  database,  and  to  enable  various  future  utilization  to  get  maximum  benefits  from  the  outputs  that  are 
currently  being  designed.  Distance  education  experts,  such  as  Bates  (1996)  and  Moore  and  Kearsley  (1996),  have 
pursued  theories  and  implementation  based  on  systemic  approaches  maybe  because  they  understand  the  enormous 
influence  of  systemic  characteristics  of  cyberspace.  As  the  space  for  distance  education  becomes  more  like  complex 
social  systems,  understanding  of  systemic  perspectives  and  theories  will  become  a necessary  part  of  educational 
technology. 

Systemic  Characteristics  of  Constructivism 

The  rise  of  constructivism  in  educational  technology  laid  a foundation  for  a better  understanding  of  basic 
concepts  and  a philosophy  of  systems  science.  Constructivism  is  a learning  theory  that  does  not  provide  necessary 
methodologies  to  challenge  mechanical  design  approaches  and  implementation  trends  or  resolve  problems  that  have 
dominated  this  field.  Constructivistic  learning  theories  themselves  cannot  provide  practical  solutions  to  various 
problems  that  require  decision  making  on  design.  For  example,  what  kinds  of  decisions  can  be  made  based  on  what 
standards  for  a learning  community  in  which  extremely  conflicting  values  among  learners  exist?  If  learning 
materials  are  extremely  transformative,  how  can  we  provide  them  to  learners  so  that  they  can  utilize  these  materials? 
Systemic  approaches  and  methodologies  will  enable  constructivistic  learning  theories  to  emerge  as  practical 
solutions  in  educational  systems. 

Constructivism  enhances  interest  in  the  learning  environment,  learning  community  design,  learner-centered 
instructional  theories  and  ill-defined  learning  domains,  thus  increasing  the  necessity  for  theories  to  design  these 
systems.  Bopry  (1999)  and  Knuth  and  Cunningham  (1993)  discussed  the  characteristics  of  constructivistic  design 
based  on  systemic  approaches. 

Bopry  (1999)  discussed  the  meaning  of  autonomous  systems  theory,  one  stream  of  second-order 
cybernetics  theories  in  educational  technology,  particularly  in  implementation  of  constructivistic  educational 
technology.  He  took  the  main  concepts  of  second-order  cybernetics  - (1)  structural  determination,  (2)  structural 
coupling,  (3)  effective  action  and  (4)  organizational  closure  - linked  them  with  constructivistic  concepts  and 
suggested  them  as  supporting  concepts  for  constructivistic  educational  design. 

Knuth  and  Cunningham  (1993)  designed  the  following  seven  principles  based  on  the  hypothesis  that 
learners  are  observing  systems  with  structural  determination,  informational  closure,  and  structural  coupling:  (1)  All 
knowledge  is  constmcted,  and  learners  experience  relative  constmctedness;  (2)  Multiple  worlds  are  possible;  thus 
multiple  perspectives  are  possible;  (3)  Knowledge  is  an  effective  action  and  a process;  (4)  Learning  is  embedded  in 
the  structural  coupling;  (5)  Knowledge  is  not  dependent  on  signs,  which  are  languages;  (6)  Worldviews  are 
searchable  and  are  changed  through  tools  and  means;  and  (7)  Our  knowing  about  how  to  know  is  the  most  ultimate 
human  achievement. 
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In  constructivism,  instmctional  design  or  educational  design  is  considered  as  a process  of  designing 
complex  social  systems  into  a learning  environment  or  learning  community,  structured  by  various  core  elements  and 
supporting  elements,  rather  than  simple  design  of  instructional  programs  design.  When  understanding  various 
emerging  natures  of  a social  system,  expressed  during  the  process  of  maintenance  and  growth  as  an  integrated 
complex  social  system  with  various  components,  people  better  appreciate  that  systems  theory  can  provide  basic 
concepts  and  methodologies  for  design,  such  as  dialogue,  languages  and  culture  that  are  necessary  for  the 
optimization  of  learning. 

New  Direction  for  Educational  Technology 

This  paper  reviewed  the  main  trends  of  systems  science  and  constmctivism  and  multimedia  environment 
that  are  gaining  more  importance  in  educational  technology,  in  relation  to  systems  science.  The  results  suggest  that 
systems  science  still  provides  essential  perspectives  and  methodologies  to  educational  technology.  The  most 
meaningful  insight  enabled  by  systems  science  is  that  educational  technology  has  various  types  of  educational 
situations  and  systems,  and  each  system  is  unique.  Therefore,  various  methodologies  and  educational  tools  should 
be  developed  that  are  appropriate  for  various  types  of  systems,  and  a macro  knowledge  base  should  be  created  and 
shared  within  the  educational  technology  society.  Based  on  these  lessons  from  systems  science,  new  alternative 
directions  for  educational  technology  can  be  suggested.  The  essence  of  the  new  direction  is  that  educational 
technology  has  so  far  pursued  prescriptions  to  resolve  educational  issues  and  problems;  however,  it  should  focus 
more  on  proscription  (that  is  design)  in  the  future. 

1.  Situations-specific  problem-solving  methods  and  design  should  be  pursued.  Systems’  uniqueness 
suggests  that  educational  systems  and  instmctional  systems  design  should  be  created  to  be  more  specific  to  different 
situations  within  each  community.  Each  community  is  unique,  and  thus  each  needs  a unique  design;  therefore,  one 
design  developed  for  a specific  community  cannot  be  applied  to  another  community  without  reflection  and 
alterations.  Fortunately,  there  have  been  substantial  efforts  to  better  understand  the  influence  of  ‘situations’  and 
develop  theories  and  models  for  instructional  systems  design  in  accordance  with  this  uniqueness.  In  particular, 
constructivistic  psychology  features  the  fact  that  learning  process  and  knowledge  constmction  is  decided  by  generic 
characteristics  of  the  environment  of  the  learners.  Learning  environments  include  every  physical,  situational  and 
human  factor  that  may  affect  learning  processes  and  performance  following  this  learning  (Vazquez-Abad  & Winer, 
1992).  The  systems  approach  pursuing  situation-specific  designs  considers  dynamics  of  more  comprehensive 
situations  and  permits  more  independence  from  design  procedures  and  more  creativity  (Banathy,  1987; 
Romiszowski,  1981;  Rowland,  1993).  As  a result,  it  enables  a more  flexible  design.  Willis  (1995)  suggested  the 
R2D2  model  (Recursive,  reflective  design  and  development  model)  as  an  instmctional  systems  design  process, 
differentiated  from  traditional  ISDs,  and  it  can  be  acknowledged  as  a method  that  can  produce  situation-specific 
designs  by  a recursive  and  non-linear  design  process,  breaking  away  from  existing  linear  and  step-oriented  models. 

2.  Design  approaches  and  methodologies  and  tools  suitable  for  the  target  systems  should  be  selected. 
Based  on  the  perspectives  that  each  system  is  unique,  very  different  specific  systems  techniques  and  methodologies 
should  be  developed  to  deal  with  the  problems  that  emerge  in  different  types  of  systems.  Flood  & Jackson  (1991), 
for  example,  recommended  operation  research,  systems  analysis  or  systems  engineering  for  simple -unitary  situations 
and  take  interactive  plans  and  soft  systems  methodology  for  complex-pluralistic  situations.  They  also  suggested 
taking  design  methodologies,  such  as  Banathy ’s  model,  for  purpose-seeking  systems  and  heuristic  systems  based  on 
Banathy’s  classification.  But  they  also  argued  that  this  design  methodology  is  not  suitable  for  coercive  systems 
where  their  sub  systems  do  not  share  common  interests  and  cannot  have  genuine  agreements,  since  they  have 
conflicting  values,  means  and  goals.  I agree  with  this  argument,  since  Banathy’s  design  methodology  is  based  on  a 
very  idealistic  systems  approach  which  essentially  ignores  conceptual  and  methodological  considerations  on  conflict 
and  power  dynamics.  The  methodology  of  second-order  cybernetics  can  provide  valuable  approaches  to  learning 
systems  based  on  multimedia  and  cyber  space  and  development  and  operation  of  educational  programs  that  are 
learners -driven  open  systems. 

3.  Expansion  of  participatory  design  and  organizational  learning  abilities  for  design  should  be  pursued. 
When  all  stakeholders  are  considered  as  learners,  breaking  away  from  the  existing  simple  perspective  of  a learner, 
defining  those  learners  is  critical  for  learning  systems  design.  In  order  to  design  human  activity  systems,  such  as 
learning  or  instructional  systems  through  participatory  design  (user  = designer),  even  development  of  organizational 
capabilities  is  needed.  Obtaining  individual  or  group  ability  for  designing  participation  is  a prerequisite  stage  for 
design  performance  and  it  includes  developing  design  thinking  and  obtaining  the  abilities  of  using  approaches, 
methods  and  tools  appropriate  for  systems  design  (Banathy,  1991;  Lee,  1995). 

4.  Pluralistic  design  approaches  should  be  pursued.  Systems  approaches  provide  pluralistic  perspective  of 
“and/  both’’,  rather  than  unitary  perspectives  of  “either/of’.  Since  there  can  be  a variety  of  solutions  to  changing 
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conditions  of  the  environment,  researchers  do  not  need  to  prescribe  a single  strategy  or  method  and  excessively 
adhere  to  the  execution  of  this  prescription.  Banathy  argues  that  if  systems  design  occurs  within  an  existing  system 
context,  we  cannot  put  the  efforts  on  solely  redesigning  it  while  giving  up  on  the  existing  system;  rather  we  also 
need  to  pay  attention  to  the  here  and  now  so  that  the  current  system  can  enhance  its  efficiency  and  effectiveness. 


Concluding  Remarks  and  Suggestions 

Educational  technology  is  receiving  demands  for  decisions  based  on  learning  perspectives,  human  learning 
and  social  systems  design  rather  than  designs  based  on  instructional  perspectives.  Now  we  can  find  the  real  value  of 
educational  technology  not  in  instmctional  design  anymore  but  in  creating  a supportive  environment  for  the  learners 
to  establish  their  own  knowledge  and  experience.  Therefore,  it  should  shed  more  light  on  ill-defined  problems, 
which  are  fundamentally  problems  of  open  systems.  Then,  educational  design,  as  suggested  by  Bopry  (1999),  will 
be  a more  appropriate  term  than  instructional  design.  In-depth  understanding  of  systems  theories  can  provide 
powerful  methodologies  for  educational  technology  to  resolve  learning  and  education  problems  which  are  ill-defined 
and  complex  because  systems  perspective  and  methodology  provide  foundations  needed  for  defining  the 
characteristics  of  systems  to  be  dealt  with  by  educational  technology.  More  specifically,  they  identify  the  natures  of 
sub-elements  and  their  functions  and  provide  supports  for  future  designs. 

However,  it  should  be  made  clear  that  we  should  break  away  from  the  traditional  practice  of  adhering  to 
‘systematic’  and  ‘systemic’  comparison.  The  main  streams  of  systems  science  are  not  much  involved  in  comparison 
between  systematic  and  systemic  characteristics  of  a system,  but  rather  classify  the  system  types  according  to  the 
combination  of  different  variables  based  on  the  systemic  concept,  understand  them  and  suggest  suitable 
methodologies  to  each  unique  system  type.  As  previously  discussed  in  New  Direction  for  Educational  Technology, 
it  will  be  more  appropriate  to  focus  on  identifying  system  types  and  solving  problems  through  methodologies 
suitable  to  the  types. 

Current  educational  technology  is  undergoing  a lot  of  changes  and  new  trials.  New  concepts  and  theories 
have  not  garnered  full  support  yet,  but  various  new  trials  are  emerging  all  around  the  world.  Since  traditional 
educational  technology  with  analysis,  absolutism,  and  reductionism  orientations  was  born  and  developed  in  the  west, 
especially  in  the  United  States,  the  other  educational  technology  circles  tended  to  follow  the  thinking  and  theories  of 
western  researchers  without  sincere  and  deep  reflections.  However,  it  is  high  time  for  the  other  educational 
technology  circles  to  break  away  from  this  vicious  cycle  and  begin  as  initiators  and  real  partners  in  the  educational 
technology  academia  for  the  following  two  reasons.  Eirst,  the  advent  of  more  complex  and  a dynamic  educational 
environment  that  cannot  be  properly  explained  or  resolved  by  traditional  paradigms  - for  example,  hypermedia, 
cyber  and  constructivistic  learning  environments.  The  unknown  world  is  waiting  to  be  tapped,  but  nobody  has 
provided  the  right  answers,  even  Gagne  and  Merrill.  Researchers  in  other  circles  are  expecting  someone  in  the  west 
to  suggest  right  answers  for  them  and  try  to  understand,  quote  and  learn  their  models  as  if  they  are  perfect,  which  is 
not  a desirable  approach.  Synthesizing  the  changes  of  postmodernism,  constructivism  and  systems  science,  it 
becomes  clear  that  western  academia  itself  acknowledges  that  its  research  was  traditionally  too  reductive  and 
analytical  and  now  it  is  trying  to  embrace  new  paradigms. 

Second,  the  systems  paradigm  is  in  some  sense  more  likely  oriental  thinking  and  methodology.  The 
traditions  of  oriental  academia  are  based  on  cosmological  and  wholistic  thinking,  which  already  have  become  the 
main  issues  of  systems  theory.  The  same  is  true  of  the  oriental  outlook  on  the  universe,  oriental  philosophy,  and 
oriental  medical  science.  The  oriental  educational  technologies  already  have  laid  the  generic  foundation  for  systems 
study.  If  they  try  to  reflect  these  generic  qualities  of  thinking  and  values  within  themselves,  tap  these  opportunities 
and  combine  them  to  their  academic  research,  they  will  be  able  to  become  the  co-partners  of  western  researchers  in 
the  educational  technology  circle  in  the  next  century. 
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Abstract 

This  paper  examines  the  use  of  interactive  videoconferencing  technology  for  providing  cross-cultural 
exposures  to  culturally  and  racially  homogeneous  K-12  leaning  environments.  Through  the  case  of  International 
Studies  for  Indiana  Schools  (ISIS),  the  authors  provide  an  instance  of  such  technology  integration.  The  present  state 
and  activities  of  ISIS  are  discussed  as  well  as  the  research  plan  on  ISIS. 

Introduction 

International  Studies  for  Indiana  Schools,  also  known  as  ISIS,  is  a video  distance-learning  (VDL)  program 
that  uses  interactive  video  technology  to  connect  K-12  schools  and  community  groups  in  Indiana  with  Indiana 
University  (lU)  international  students,  scholars,  and  specialists.  In  1995,  ISIS  was  created  jointly  by  the  Office  of 
International  Programs  and  the  Center  for  Research  on  Learning  and  Teaching.  Using  presenters  from  Indiana 
University's  many  world  area  studies  centers,  this  unique  outreach  program  is  realized  by  means  of  Vision  Athena, 
an  interactive  video  project  sponsored  by  Ameritech's  non-profit  Corporation  for  Educational  Communications. 

The  current  state  of  ISIS 

The  main  goal  of  the  ISIS  program  is  to  provide  Indiana  students  with  access  to  other  cultures  through  live 
interaction  with  the  speakers  from  those  cultures.  With  its  large  number  of  international  students,  lU  serves  as  a 
wonderful  pool  of  speakers.  When  interested  teachers  contact  the  director  of  ISIS,  she  identifies  possible  presenters 
and  together  they  design  and  develop  the  program  for  a particular  classroom.  The  program  is  tailored  according  to 
the  nature  of  the  specific  topic  and  the  particular  class  context.  For  example,  when  a high  school  French  teacher 
requested  a program  on  Francophone  Africa,  the  coordinator  of  ISIS  found  an  ethnomusicology  graduate  who 
studied  Mali  music  and  set  up  an  interactive  program  on  the  influence  of  French  colonization  on  Mali  music  for  the 
French  class.  ISIS  topics  include  a range  of  information  and  issues  related  to  foreign  cultures  and  countries. 
Examples  of  past  programs,  to  name  a few,  include  Tea  Ceremony  of  Japan  (for  5'*’  grade).  The  Culture  of 
Afghanistan  (for  lO'*’  grade  social  studies)  &n&Daily  life  in  Korea  (for  7‘*'  graders).  ISIS  uses  two-way  interactive 
video  technology  (lAV)  through  fiber  optic  lines  or  Internet  protocol  (IP)  connection. 

Need  for  cross-cultural  understanding 

Since  the  tragedy  of  September  1 L’,  many  Americans,  along  with  the  rest  of  the  world,  have  come  to 
realize  the  importance  of  fostering  mutual  respect  for  peoples  of  other  nations  and  their  cultures.  With  recent 
developments  in  world  affairs,  many  educators  feel  an  urgent  need  to  increase  the  inclusion  of  international  content, 
issues  and  perspectives  into  the  existing  curriculum.  An  effective  international  education  curriculum  consists  of 
more  than  simple  facts  and  figures  about  nations  and  their  relations  with  one  another;  it  also  encourages 
understanding  of  cultural  differences  and  similarities,  tolerance,  and  a globally  interdependent  view  of  the  world 
(Pinhey,  2001).  Such  curriculum  can  help  students  grow  beyond  their  original  cultural  parameters,  and  thus  achieve 
a sense  of  identity  and  selfhood  that  is  at  once  individualized  and  universalized  (Kim,  2001). 

Over  the  past  30  years,  numerous  non-profit  organizations  and  grassroots  groups  across  the  United  States 
have  organized  to  further  the  integration  of  international  studies  into  their  existing  curricula  (Pinhey,  2001).  In 
California,  for  example,  the  Stanford  Program  on  International  and  Cross-Cultural  Education  (SPICE)  has  supported 
efforts  to  internationalize  elementary  and  secondary  school  curricula  by  linking  the  research  and  teaching  at 
Stanford  University  with  children  in  K-12  schools  (http://spice.stanford.edu/about/index.html).  Tye  (2002)  points 
out  that  most  educators  who  have  become  involved  in  such  efforts  share  the  following  general  goals:  (a)  broadening 
students’  horizons,  including  helping  them  to  see  world  issues  from  a variety  of  perspectives;  (b)  building  critical 
thinking  skills;(c)  better  preparing  young  people  for  productive  lives  in  a nation  that  belongs  to  an  increasingly 
interdependent  world  community. 

Once  general  awareness  of  the  need  for  such  programs  has  been  established,  these  goals  should  be 
translated  to  be  realized  in  more  localized  contexts.  It  is,  therefore,  very  important  that  the  educators  of  Indiana 
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should  make  serious,  immediate  efforts  to  include  international  studies  for  K-12  schools  in  rural  Indiana.  Because  of 
their  isolation  and  relative  homogeneity  of  their  cultural  environments,  without  some  effective  intervention  the 
students  in  rural  Indiana  schools  are  at  a relative  disadvantage  in  preparing  themselves  for  the  global  age. 

For  the  state  of  Indiana,  the  pressure  to  provide  accurate  international  programs  for  global  education  is  not 
only  because  of  the  state’s  population  in  relation  to  external  population.  According  to  documents  provided  by 
Indiana  University  International  Resource  Center  (lUIRC),  Indiana  has  experienced  a recent  increase  in  numbers  of 
immigrants,  and  this  has  occurred  especially  in  the  traditionally  homogeneous  smaller  communities.  New 
manufacturers  and  industries  have  contributed  to  this  phenomenon.  The  findings  of  the  lUIRC  show  that  the  small 
towns  into  which  a large  number  of  immigrants  have  moved  are  facing  new  challenges  in  all  sectors  of  community 
life.  More  often  than  not,  many  of  the  original  residents  of  these  communities  do  not  have  sufficient  understanding 
of  cross-cultural  issues  to  welcome  the  new  residents.  These  internal  conditions  within  Indiana,  as  well  as  the 
external  factors  of  understanding  the  United  States’  place  in  a globalizing  world,  call  for  more  international  content 
in  the  Indiana  school  curriculum.  If  done  well,  such  international  curriculum  will  help  to  prepare  our  children  for  a 
multi-cultural  and  global  society. 

One  of  the  effective  ways  of  bringing  the  world  to  these  rural  schools  is  using  interactive 
videoconferencing  technology.  Videoconferencing  technology  is  relatively  simple  to  use,  yet  highly  effective  when 
appropriately  integrated  into  the  curriculum.  The  interactive  and  synchronous  nature  of  the  technology  makes  it 
especially  useful  in  teaching  about  different  cultures.  Using  this  technology,  a video  image  of  the  speaker  is  received 
on  the  classroom  television  and  the  speaker  can  tell  about  their  countries  and  the  students  can  interact  with  the 
speaker  during  the  presentation. 

In  this  regard,  the  ISIS  project  has  received  much  positive  feedback  from  teachers  and  the  students  who 
have  used  the  program  in  their  curriculum  and  many  new  requests  have  been  received.  In  February  2002,  ISIS  did  a 
session  with  a senior  high  school  world  history  class  on  life  and  culture  of  Afghanistan.  The  speaker  was  a female 
law  student  at  lU  who  had  dual  citizenship  for  Afghanistan  and  the  United  States.  The  followings  are  actual 
comments  from  the  response  papers  of  three  students  about  the  program: 

The  distance  learning  that  was  on  Friday  was  an  eye  opener  and  a mouth  dropping  experience.... 

Fve  seen  interviews  of  Afghanistan  women  on  television,  talking  about  what  it  was  like  to  live 
there.  Just  like  the  interviews  on  television,  I thought  talking  to  Amida*(the  name  has  been 
changed)  would  be  the  same  feeling,  but  it  wasn’t.  It  was  very  different.  Actually  being  able  to  see 
her  and  the  fact  knowing  that  she  could  see  us,  made  me  think  I wasn  ’t  just  watching  TV,  but 
instead  having  a face-to-face  conversation. 

I was  shocked  by  how  much  information  she  gave  us  in  an  hour.  She  had  covered  racial  profiling 
to  a new  government  in  Afghanistan.  I am  very  glad  that  I had  the  chance  to  listen  to  this  amazing 
point  of  view. 

After  this  distance  learning  I got  a better  understanding  of  the  relationships  between  the  US  and 
Afghanistan.  I also  now  understand  how  natives  of  the  Afghanistan  homeland  who  live  in  the  US 
must  feel.  Thanks  to  this  distance  learning.  I'm  not  so  narrow  minded  about  the  world  today. 

From  August  15,  2000  to  August  14,  2001,  there  were  approximately  80  requests,  in  response  to  which  65 
programs  were  carried  out.  This  year  the  number  and  variety  of  requests  has  increased.  For  example,  in  addition  to 
typical  requests  for  a single  cultural  program,  some  teachers  have  asked  for  a series  of  programs  or  programs  to 
supplement  foreign  language  courses. 

Interactive  Videoconferencing 

In  the  researches  on  telecommunication,  researchers  have  shown  that  telecommunications  help  bringing 
students  and  teachers  out  of  the  isolation  from  their  schools  into  a real  world.  Telecommunications  not  only  helps 
curricula  of  subjects  like  social  studies  by  providing  more  access  to  current  information  but  also  expand  the 
communication  beyond  the  classrooms  and  schools,  bringing  together  people  with  different  perspectives  (Roberts  et 
al,  1990).  In  many  cases,  the  interactive  videoconferencing  (lAV)  shares  the  same  benefits  as  those  of 
telecommunication.  Learning  about  other  cultures  can  be  done  most  effectively  with  some  direct  experiences  with 
that  culture.  However,  when  such  option  is  not  possible  like  in  the  cases  of  rural  Indiana  where  white  populations 
exceed  95%  easily,  one  can  turn  to  appropriate  technology  that  can  simulate  the  face-to-face  interaction  most 
closely.  lAC  can  serve  as  such  technology.  The  Guide  to  Distance  Learning  points  out  that  the  real-time  broadcast 
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of  video-based  instruction  is  the  closest  replication  of  the  traditional  classroom  that  distance-learning  technology 
offers  (Yoakam,  1995).  The  fact  that  lAV  is  low  cost  and  closely  resembles  real  human  interaction  makes  the  use  of 
the  technology  for  the  content  of  international  studies,  a perfect  fit.  Opportunities  for  learners  to  express  their  own 
points  of  view  explaining  the  issues  in  their  own  words  and  to  formulate  opposing  or  different  arguments  have 
always  been  related  to  deep-level  learning  and  the  development  of  critical  thinking. 

Synchronous  communication  by  voice  and,  better  still,  vision  overcomes  the  prolonging  of  exchanges,  but 
of  course  requires  immediate  response.  However,  if  the  presenter  is  not  spontaneous  enough,  the  characteristic  of 
immediacy  can  work  against  him/her.  Long  pauses  in  conversation  are  not  acceptable  and  hence,  ‘thinking  on  one’s 
feet’  is  necessary  in  synchronous  communication  (Mason,  1995). 

In  the  ISIS  sessions,  it  was  clear  to  see  how  both  the  presenter  and  the  students  benefited  from  the 
interactivity  of  the  medium.  The  session  has  to  be  a dialogue  between  the  representatives  of  two  cultures,  not  a one- 
way lecture.  The  facilitators  of  the  sessions  always  encouraged  the  presenter  to  ask  and  solicit  questions,  because  we 
truly  believed  having  the  synchronous  interaction  with  the  representative  from  another  culture  will  not  only  make 
the  sessions  more  interesting  and  exc  iting  but  will  also  foster  further  interest  in  that  culture  in  the  long  run.  In  this 
regard,  it  is  very  important  to  solicit  as  much  input  from  students  during  the  course  of  presentation  as  possible.  By 
not  providing  the  presenters  with  any  questions  beforehand  but  giving  them  at  the  real  time,  the  students  could  get 
more  honest,  first-hand  answers.  Because  the  questions  are  not  screened  beforehand,  there  are  times  when  children 
ask  or  answer  questions  that  might  be  considered  “impolite”  without  any  intentions  of  doing  so.  It  is  important  for 
the  presenter  in  this  situation  to  understand  that  such  incidences  are  usually  due  to  the  lack  of  sufficient  exposure  to 
other  cultures.  As  a matter  of  fact,  we  take  such  cases  as  an  evidence  of  why  program  like  ISIS  is  really  important  to 
the  students.  Some  of  the  useful  tips  for  facilitating  ISIS  sessions  are  as  follows: 

• Invite  questions  from  audience  after  each  topic 

• Show  great  enthusiasm 

• Focus  only  on  3-4  main  points 

• Change  activities  every  5-10  minutes 

• Ask  audience’s  opinion  on  the  topics 

The  characteristics  pertaining  to  the  particular  audience  such  as  the  grade  level  is  also  a very  important  point  of 
consideration  because  the  effective  level  of  interactivity  varies  accordingly. 

The  research  on  ISIS 

In  spite  of  a short  but  successful  history  in  terms  of  increasing  demand  and  anecdotal  evidence  of 
satisfaction,  no  formal  research  has  been  conducted  on  if  these  programs  actually  have  an  impact  on  the 
understanding  and  the  attitudes  towards  other  cultures  of  rural  Indiana  students.  Also  in  the  field  of  Instructional 
Technology,  where  the  issues  of  technology  integration  have  received  full  attention,  there  has  not  been  much 
research  done  specifically  on  the  use  of  interactive  videoconferencing  for  instructional  purposes.  The  previous 
researches  on  videoconferencing  have  mainly  been  limited  to  design  and  implementation  tactics  on  a very  general 
level. 

A recent  effort  has  been  made  to  study  the  possible  integration  of  technology  into  multicultural  education 
(Marshall,  2001).  It  seems  that  the  time  is  ripe  for  looking  at  the  use  and  the  impact  of  videoconferencing  in 
international  studies  curriculum  for  rural  K-12  students. 

The  purpose  of  this  study  is  to  obtain  a better  understanding  of  the  situations  and  conditions  affecting  the 
cross-  and  multicultural  awareness  of  students  in  rural  Indiana  and  the  role  of  the  videoconferencing  technology  in 
such  situations.  Understanding  others’  perspectives  is  a crucial  component  of  multicultural  knowledge  development 
as  well  as  of  cultural  identity  construction;  that  is,  how  one  sees  one’s  place  in  the  world.  Videoconferencing 
technology  can  serve  as  an  effective  solution  to  rural  (or  even  urban)  isolation  from  people  from  other  national 
backgrounds  by  bringing  people  and  artifacts  of  other  cultures  into  classrooms.  Marshall  (2001)  points  out  that 
technologies  such  as  interactive  videoconferencing  can  be  used  to  promote  positive  self-concept  and  foster  positive 
relationships  between  people  from  diverse  backgrounds. 

As  mentioned  previously,  ISIS  is  affiliated  with  two  organizations:  International  Programs  and  the  Center 
of  Research  and  Learning  in  Technology.  There  are  many  parties  involved  in  these  videoconferencing  programs: 
the  Center  for  Interactive  Learning  and  Collaboration  (CILC),  Vision  Athena,  the  participating  schools  and  students, 
as  well  as  the  ISIS.  With  extensive  experience  in  the  use  of  instructional  technology  and  its  design  issues,  the 
research  team  will  work  closely  with  both  the  school  and  campus-based  parties  involved.  Once  the  impact  of  these 
international  programs  is  closely  examined,  the  research  team  will  be  able  to  provide  more  information  that  will 
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contribute  to  opening  more  doors  to  the  world  for  our  rural  students.  In  this  sense,  interactive  videoconferencing  is 
the  technology,  which  not  only  provides  otherwise  impossible  contact  with  various  cultures  but  also  connects  people 
in  a way  that  closely  resembles  personal  interaction. 

Research  Questions 

Two  interconnected  questions  guide  this  research: 

1.  What  are  the  reactions  of  Indiana  middle  school  students  from  racially  homogeneous  environments  when 
introduced  to  the  series  of  international  studies  programs  through  interactive  videoconferencing 
technology? 

2.  How  is  interactive  videoconferencing  technology  used  as  a medium  for  bringing  global  context  to  racially 
isolated  students? 

A description  of  the  research  process 

In  order  to  examine  the  interrelated  factors  that  influence  the  multi-cultural  awareness  in  the  rural  K- 12 
learning  environment,  qualitative  case  study  is  appropriate.  The  impact  of  the  program  which  will  be  implemented 
for  a whole  school  year  cannot  be  accurately  analyzed  by  simple  questionnaires  or  surveys,  from  the  point  of  both 
the  content  and  technology.  An  ethnographic  approach  that  takes  into  consideration  of  various  aspects  as  well  as 
giving  a thick  description  can  provide  a detailed  picture. 

Site  selection 

Two  school  sites  in  rural  Indiana  have  been  selected  by  purposive  sampling.  They  meet  the  following 
criteria:  (1)  both  sites  are  social  studies  classes  in  junior  high  schools  in  rural  Indiana,  (2)  the  social  studies  teachers 
have  agreed  to  having  the  international  studies  program  incorporated  as  an  integral  part  of  their  social  studies 
curriculum  throughout  the  school  year  of  2002-2003,  and  (3)  classrooms  are  in  culturally  isolated  rural  schools  with 
a history  of  a very  high  level  of  racial  homogeneity  with  the  current  student  population  being  at  least  95%  white 
Both  sites  have  both  have  a racial  profile  of  100%  Caucasian  (Indiana  Census,  2001).  Although  the  state  average  of 
white  dominance  is  87.5%,  there  are  67  out  of  a total  of  92  counties  where  at  least  95%  of  residents  are  white. 
Sampling  of  participants 

Upon  the  suggestion  of  the  participating  teachers,  two  l'^  grade  social  studies  classes  have  been  selected. 
The  teachers  felt  that  these  two  classes  provide  a good  range  of  students  with  different  performance  levels. 

Data  collection 

The  data  for  this  study  will  be  comprised  of  observations,  interviews  and  document  analysis.  They  will  be 
collected  at  both  sites  between  October  2002  and  May  2003.  The  Table  7. below  provides  the  specific  method  of  data 
collection. 


Observation 

♦ 

Classrooms  (regular) 

♦ 

Twice  a month  for  50  min. 
each 

♦ 

ISIS  sessions 

♦ 

6-8  sessions  for  45  -60  min. 
each 

♦ 

The  school  sites  (hallways, 
cafeterias  & etc) 

♦ 

Once  a week 

Interviews 

♦ 

Students 

♦ 

Twice  (individual/focus 
group) 

♦ 

Teachers 

♦ 

Three  times 

♦ 

ISIS/CILC  personnel 

♦ 

Once 

♦ 

lU  presenters  (6-10) 

♦ 

Once 

Document  Analysis 

♦ 

Textbooks 

♦ 

Occasionally 

♦ 

Other  instructional  materials 

♦ 

Occasionally 

♦ 

Reflection  papers 

♦ 

6-8  tines,  once  after  each 
session 

Field  notes 

♦ 

Informal  conversations  with 
teachers 

♦ 

Frequently 

Table  1.  The  method  of  data  collection 


As  shown  above,  the  people  who  are  involved  in  the  design  and  implementation  of  the  ISIS  program  will 
also  be  interviewed  aside  from  interviewing  the  students  and  the  teachers.  This  group  of  interviewees  will  consist  of 
the  coordinator  of  ISIS,  officials  of  CILC,  and  the  international  student  presenters  who  participated  in  ISIS  programs 
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at  each  site.  The  purpose  of  interviewing  these  individuals  is  to  compare  their  vision,  expectations  and  reactions  to 
the  program  to  those  of  the  recipients  of  the  program;  that  is,  the  students  and  the  teachers.  Bringing  such  multiple 
perspectives  will  provide  a deeper  understanding  of  the  impact  of  such  program  and  the  technology  used.  During  the 
study,  literature  reviews  of  the  purposes  and  goals  of  global  studies  will  be  continuously  carried  out.  In  addition, 
other  types  of  international  studies  curricula  from  a variety  of  locations  will  be  inspected.  All  interviews  will  be 
conducted  face-to-face. 

Data  Analysis 

Critical  ethnography  (Carspecken,  1996)  and  emancipatory  collaborative  research  methodology  (Lather, 
1991)  will  provide  guidance  for  the  data  analysis  of  this  study.  The  decisions  regarding  how  faithfully  the  research 
team  will  follow  the  more  detailed  suggestions  of  the  two  methodologies  will  be  made  through  continuous  dialogue 
with  the  participants  of  the  study.  The  software  for  qualitative  study,  NVivo,  will  be  used  for  coding  and  analysis 
process. 

Conclusion 

By  introducing  the  International  Studies  for  Indiana  Schools  (ISIS),  this  paper  has  attempted  to  examine  a 
case  of  videoconferencing  technology  integration  for  cross-cultural  education  in  rural  K- 12  environments.  The 
design  plan  of  the  long-term  research  on  ISIS  introduced  in  this  paper  is  a work  in  progress.  Through  the 
collaborative  research  design  with  the  teachers,  the  ISIS  coordinators  and  the  research  team,  the  result  of  this  study 
will  be  able  to  provide  helpful  guidelines  for  further  use  of  the  technology  for  bringing  the  world  into  the  classroom. 
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Abstract 

Student/faculty  mentoring  models  have  shown  promise  as  an  effective  technique  for  integrating  technology 
into  courses  and  classes  among  teacher  educators.  (Kortecamp  & Cronginger,  1995;  Mergendoller,  1994). 
Mentoring  is  reported  as  a valuable  means  in  assisting  teacher  education  faculty  in  integrating  technology  in  their 
courses  (Thompson,  1995).  It  has  emerged  as  a promising  staff  development  model  to  help  teacher  education  faculty 
learn  and  use  technology  (Kortecamp  & Croninger,  1996;  Abate,  2001;  Thompson  & Schmidt,  1995).  However, 
little  is  known  about  how  and  why  mentoring  programs  work  to  assist  faculty  members  in  integrating  technology. 

This  research  investigated  how  and  why  the  student/faculty  mentoring  program  in  the  Department  of 
Curriculum  and  Instruction  at  Iowa  State  University  works  as  an  effective  staff  development  approach  to  assist 
teacher  education  faculty  in  integrating  technology.  It  explored  the  characteristics  of  successful  mentoring 
relationship  in  the  ISU  student/faculty  mentoring  program,  the  roles  of  mentors  and  mentees,  and  the  perspectives 
from  both  faculty  members  and  student  mentors  about  how  and  why  the  mentoring  program  assists  faculty  in  using 
and  integrating  technology.  A qualitative  case  study  approach  is  used  to  describe  and  analyze  the  ISU 
student/faculty  mentoring  program.  Three  graduate  students  and  five  faculty  mentees  who  joined  the  mentoring 
program  in  the  Fall,  2000  participated  in  the  research  study.  Data  such  as  interviews,  documents,  and  observation 
were  collected  to  describe  and  analyze  the  characteristics  of  successful  mentoring  relationships,  roles  of  mentors  and 
mentees,  and  perspectives  of  both  faculty  mentees  and  student  mentors  about  mentoring.  Findings  from  the  research 
study  indicate  that  using  student  mentors  to  assist  faculty  members  to  integrate  technology  into  teaching  is  a 
promising  approach  for  higher  education.  Results  will  provide  information  on  designing  an  effective  mentoring 
model  for  higher  education  faculty  staff  development. 

ISU  Student/faculty  Mentoring  Program 

The  establishment  of  the  student/faculty  mentoring  program  in  the  Department  of  Curriculum  and 
Instruction  at  Iowa  State  University  can  be  traced  back  to  1991.  The  purpose  of  the  ISU  student/faculty  mentoring 
program  is  to  assist  faculty  members  as  they  learn  about  and  use  technology.  It  provides  teacher  education  faculty 
with  an  opportunity  to  learn  about  technology  in  a non-threatening  learning  environment.  The  ultimate  goal  of  the 
mentoring  program  is  for  faculty  members  to  become  confident,  independent  computer  users  who  model  effective 
uses  of  technology  in  their  courses. 

In  1991,  the  department  began  providing  office  computers  to  all  faculty  members  who  were  interested  in 
using  technology  in  their  classes.  With  grant  funding,  the  department  continues  to  upgrade  faculty  computers  and  to 
provide  software  for  faculty  to  use  in  their  courses.  As  access  to  computers  increased  for  faculty,  the  need  to  support 
their  efforts  to  learn  about  using  the  technology  grew.  Hence,  graduate  students  and  faculty  members  voluntarily 
paired  up  and  worked  together  on  specific  technology  projects  that  faculty  mentees  had  identified.  Thus,  faculty 
members  who  were  interested  in  using  and  learning  about  technology  had  access  to  a computer  and  support  from 
their  student  mentor. 

Since  1993,  the  department  has  connected  the  mentoring  program  with  a graduate  course  entitled 
“Technology  in  Teacher  Education  (C  1 610).  As  a part  of  the  course,  students  who  had  technology  expertis  e were 
matched  up  with  faculty  members  desiring  assistance  according  to  their  personality,  interest,  background,  and 
technology  experience  (Thompson  et  al,  1996).  Student  mentors  then  help  faculty  members  at  their  level  of 
technology  experience  and  cater  to  their  individual  needs  and  interest  (Zachariadess  & Roberts,  1995).  It  is  the 
responsibility  of  the  teacher  education  faculty  members  to  define  their  own  working  agenda,  choose  the  mentoring 
meeting  place,  and  set  up  mentoring  meeting  times.  During  the  first  meeting,  both  the  student  mentor  and  faculty 
mentee  set  up  goals  for  the  semester  and  schedule  their  meeting  times.  The  remaining  mentoring  sessions  throughout 
the  semester  are  focused  on  the  faculty  members’  needs  and  interests.  After  each  mentoring  session,  student  mentors 
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are  required  to  write  and  reflected  about  session's  accomplishments.  The  purpose  of  the  student  journals  is  to  record 
the  interaction  and  happenings  of  each  mentoring  session.  Besides  meeting  with  their  faculty  mentees  each  week, 
graduate  students  also  meet  as  a group  for  seminar  once  a week.  These  seminars  provide  opportunities  for  the 
graduate  students  to  share  their  thoughts  about  their  mentoring  experience,  to  get  help  and  suggestions  from  the 
course  instructor  and  other  students,  and  to  participate  and  discuss  articles  about  contemporary  technology  issues  in 
teacher  education.  By  the  end  of  the  course,  the  student  mentors  write  up  a case  report  that  describes  their  mentoring 
experience  and  their  perspectives  on  the  mentoring  process.  Since  establishment  of  the  mentoring  program,  an 
increasing  number  of  faculty  members  and  graduate  students  have  joined  the  mentoring  program.  In  2002,  29  or  the 
30  tenure  track  faculty  in  the  department  had  participated  in  the  mentoring  program  for  at  least  one  semester.  The 
mentoring  program  has  gradually  become  a systematic  approach  in  the  department  to  assist  teacher  education  faculty 
in  using  and  integrating  technology. 

The  ISU  student/faculty  mentoring  program  also  receives  administrative  support  from  the  Department  of 
Curriculum  and  Instruction  and  the  Center  for  Technology  in  Learning  and  Teaching.  The  chair  of  the  Department 
of  Curriculum  & Instruction  has  been  highly  supportive  of  the  mentoring  program.  Every  fall,  the  opportunity  to 
participate  in  mentoring  program  is  announced  during  a departmental  faculty  meeting.  Faculty  are  then  encouraged 
to  sign  up  for  the  program  to  receive  assistance  from  graduate  student  mentors.  It  is  also  worth  noting  the  support 
that  the  Center  of  Technology  in  Teaching  and  Learning  (CTLT)  in  the  College  of  Education  provides  for  the 
mentoring  program.  The  CTLT  is  a nurturing  environment  and  is  comprised  of  a group  of  faculty,  graduate  students, 
undergraduate  students,  and  staff  who  are  interested  in  using  technology  in  education.  The  CTLT  is  a technology- 
rich  learning  environment  that  includes  three  computer  labs,  technology  production  facilities,  and  a model  student- 
centered  classroom  with  distance  education  capabilities.  Whenever  faculty  members  have  a need  for  using 
technology  in  their  teaching,  they  can  reserve  equipment  and/or  a computer  lab.  Faculty  and  students  can  also  check 
out  hardware,  software  and  books  from  the  CTLT.  Naturally,  CTLT  provides  the  space  and  support  that  faculty  may 
need  to  incorporate  technology  integration  ideas  into  their  courses.  Both  student  mentors  and  their  faculty  mentees 
receive  assistance  from  CTLT  staff  when  encountering  hardware/software  problems  or  generating  technology 
integration  ideas  for  their  classes. 

In  summary,  the  ISU  student/faculty  mentoring  model  is  a systematic  approach  used  to  assist  teacher 
education  faculty  in  the  Department  of  Curriculum  & Instruction  in  their  attempts  to  use  and  integrate  technology. 
However,  the  mentoring  model  has  not  been  thoroughly  investigated.  Research  is  needed  to  fully  understand  how 
and  why  this  mentoring  model  might  assist  the  teacher  education  faculty  in  learning  about  technology.  Issues  such 
as  characteristics  of  successful  mentoring  relationships  and  roles  of  student  mentors  and  faculty  mentee  need  to  be 
examined  (Reinhart,  1997).  The  more  we  understand  about  the  components  of  this  mentoring  model,  the  better  we 
can  assist  teacher  education  faculty  with  technology  use  and  integration.  Therefore,  it  is  critical  to  further  investigate 
this  student/faculty  mentoring  model. 

Research  Questions 

This  research  study  is  to  investigate  how  and  why  the  Iowa  State  University  (ISU)  student/faculty 
mentoring  model  works  to  assist  teacher  education  faculty  to  use  and  integrate  technology  in  the  Department  of 
Curriculum  and  Instruction  at  ISU.  The  guiding  questions  of  this  research  study  are: 

1)  What  are  the  characteristics  of  successful  mentoring  relationships  of  the  ISU  student/faculty  mentoring 
program? 

2)  What  are  the  roles  of  student  mentors  and  faculty  mentees  in  the  ISU  student/faculty  mentoring  program? 

3)  What  are  the  perspectives  from  teacher  education  faculty  members  about  how  and  why  the  ISU 
student/faculty  mentoring  program  assists  them  to  use  and  integrate  technology? 

4)  What  are  the  perspectives  from  student  mentors  about  how  and  why  the  ISU  student/faculty  mentoring 
program  assists  teacher  education  faculty  to  use  and  integrate  technology? 

Research  Participants 

Three  student  mentors  who  were  enrolled  in  the  C I 610  during  fall  semester,  2000  and  their  faculty 
mentees  were  selected  for  this  research  study.  Participant  selection  was  based  on  the  following  criteria:  1)  Only  the 
student  mentors  and  their  teacher  education  faculty  mentees  worked  in  the  Department  of  Curriclum  & Instruction. 

2)  they  held  weekly  mentoring  meetings  3)  they  worked  collaboratively  on  technology  integration  projects  identified 
by  faculty  mentees.  A brief  profile  for  each  student  mentor  and  faculty  mentee  was  provided  in  Table  1.  All  research 
participants  were  assigned  names  for  the  sake  of  confidentiality. 
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Table  1.  Description  of  faculty  mentees  and  student  mentors 


Faculty  mentees 

Student  Mentor 

John:  assistant  professor,  teaches  courses  in 
multiculturalism 

Mary:  2nd  year  doctoral  student,  research  assistant  in 
the  Department  of  Curriculum  & Instruction,  worked 
in  a publishing  business  before  coming  to  graduate 
school. 

Tom:  assistant  professor,  teaches  courses  in 
educational  foundations 

Edward:  professor,  teaches  courses  in  special 
education. 

Jane:  a first  semester  masters  student  in  the 
Department  of  Curriculum  and  Instruction,  taught  at 
elementary  level  for  3 years  before  coming  to  graduate 
school. 

Diana:  temporary  instructor,  teaches  courses  in 
educational  foundations  and  children's  literature. 

Crystal:  1st  year  doctoral  student,  teaching  assistant 
for  the  Department  of  Curriculum  and  Instruction, 
taught  high  school  English  before  coming  to  graduate 
school. 

Cindy:  associate  professor,  teaches  courses  in  literacy 
and  reading  education 

Research  Design 

Case  study  is  a preferred  research  methodology  used  when  “how”  and  “why”  questions  are  being  posed, 
when  the  investigator  has  little  control  over  events,  and  when  the  focus  is  on  a contemporary  phenomenon  within 
some  real-life  context  (Yin,  1994).  Because  this  research  study  explores  ‘how”  and  ‘why”  the  ISU  mentoring  model 
works  to  support  and  facilitate  faculty  use  of  computer  technology,  the  case  study  approach  was  selected  to  reveal 
the  dynamic  process  of  mentoring.  Due  to  the  nature  of  the  mentoring  process,  words  rather  than  numbers  will  better 
illustrate  and  capture  the  phenomena  of  the  entire  mentoring  process  (Merriam,  1988) 

Research  Procedures 

The  three  student  mentors  and  their  faculty  mentees  met  one  hour  per  week  for  the  entire  semester.  These 
meetings  were  scheduled  by  each  mentoring  pair.  After  each  mentoring  session,  student  mentors  were  required  to 
record  in  a journal  the  progress  of  their  mentoring  session.  These  entries  included  such  details  like  achievements  and 
challenges  that  were  faced  by  the  mentoring  pairs.  Each  week,  graduate  student  mentors  also  met  for  two  hours  with 
classmates  and  the  course  instructor  to  share  information  about  their  mentoring  experiences  and  to  explore  the 
contemporary  issues  about  technology  in  teacher  education.  At  the  end  of  the  semester,  student  mentors  were 
required  to  write  a case  study  report  about  their  own  mentoring  experiences  and  perspectives  on  mentoring.  In  each 
case  report,  student  mentors  provided  a mentor/mentee  profile,  a reflection  of  their  mentoring  experience,  and  a 
summary  of  what  they  gained  from  the  mentoring  experience. 

At  the  end  of  the  semester,  the  three  student  mentors  and  their  faculty  mentees  were  interviewed  by  the 
researcher.  The  purpose  of  the  interview  was  to  gather  additional  information  about  their  mentoring  experiences. 
Interview  questions  were  e-mailed  to  interviewees  prior  to  the  interview.  Interview  questions  were  structured  so 
additional  data  about  mentoring  characteristic,  roles  and  perspectives  would  be  gathers. 

Finally,  the  researcher  interviewed  both  the  associate  director  of  the  Center  of  Technology  in  Teaching  and 
Learning  and  the  chair  of  the  Department  of  Curriculum  and  Instruction.  These  interviews  were  conducted  to  gather 
data  about  the  program  background  and  history  and  how  administration  has  supported  this  program. 

It  is  worth  to  note  that  in  order  to  have  an  insider  view  of  the  mentoring  program,  the  researcher  signed  up 
as  a student  mentor  in  the  mentoring  program  by  enrolling  in  the  course  C I 610  in  the  fall,  2000  and  fully 
participated  in  the  mentoring  program  herself. 

Data  Collection  and  Analysis 

The  purpose  of  this  study  was  to  describe  and  to  analyze  how  and  why  mentoring  is  an  effective  approach  to 
assist  teacher  education  faculty  in  improving  computer  skills  and  to  use  technology  in  their  teaching.  The  study 
aimed  at  presenting  a full  perspective  of  the  mentoring  program  from  both  faculty  mentees  and  graduate  mentors.  In 
this  descriptive  case  study,  data  such  as  student  mentors'  journals,  case  reports,  interviews  and  observations  were 
collected  and  analyzed  to  answer  the  following  research  questions. 

The  data  were  then  analyzed  by  using  the  constant  comparative  method  (Merriam,  1988).  It  has  four  main 
phases:  unitizing,  categorizing,  filling  in  patterns,  and  member  checking.  Unitizing  is  the  process  of  extracting  the 
small  pieces  of  meaningful  information  from  the  data.  In  this  research  study,  data  were  collected  from  interview, 
student  journals,  student  mentors'  case  reports,  and  class  observations.  Categorizing  is  to  place  the  units  in 
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provisional  categories.  Based  on  the  research  questions,  initial  categories  of  the  data  were  developed:  characteristics 
of  successful  mentoring  relationship,  mentor  roles,  mentee  roles,  the  perspective  of  student  mentors  about 
mentoring,  and  the  perspective  of  faculty  mentees  about  mentoring.  Additional  categories  such  as  mentoring 
objectives  were  developed  with  the  actual  data  analysis  process.  The  third  phase  is  to  fill  in  the  patterns.  After 
categorizing  data,  categories  were  reviewed  for  possible  overlaps  and  relations.  Then  the  researcher  carefully 
examined  data  of  each  category  and  the  emerging  themes  were  identified  to  reveal  the  pattern  of  the  mentoring 
process  (Merriam,  1985).  With  each  emerging  theme,  pieces  of  evidences  from  the  data  were  illustrated  to  support 
the  themes.  The  final  phase  is  to  member  checking.  This  phase  involves  taking  the  data  and  interpretations  back  to 
the  participants  who  provided  the  data  and  to  check  with  them  to  see  if  the  results  are  reported  accurately  and 
realistically  (Merriam,  1988). 

Findings  and  Discussions 

Research  Question  One 

What  are  the  characteristics  of  effective  mentoring  relationship  in  ISU  student/faculty  mentoring  program? 

There  are  many  factors  that  contribute  to  the  dynamics  of  mentoring  relationship.  All  characteristics  for  a 
successful  mentoring  relationship  identified  in  this  research  comply  with  literature  findings.  Facutly  mentees  and 
student  mentors  in  the  ISU  mentoring  program  believe  the  following  characteristics  are  essential  to  ensure  their 
effective  mentoring  relationships. 

• Time.  Time  is  the  precondition  for  the  mentoring  to  occur  (Kay,  1990).  Making  time  for  the  mentoring 
sessions  is  vital  for  the  effective  mentoring  relationships.  Mentors  and  mentees  need  time  to  establish  trust, 
credibility  and  rapport  in  the  mentoring  relationship.  Without  investing  a sufficient  amount  of  time,  a good 
and  healthy  relationship  can't  be  nurtured  during  the  mentoring  program.  Participants  in  this  research  study 
mentioned  that  setting  aside  time  to  work  together  was  very  important.  In  the  ISU  student/faculty 
mentoring  program,  student  mentors  and  faculty  mentees  were  required  to  set  aside  one  hour  each  week  to 
work  on  their  technology  project.  All  participants  made  a deliberate  effort  to  keep  their  commitment  and 
scheduled  regular  mentoring  sessions  every  week. 

• Mutual  trust  and  respect.  Mutual  trust  and  respect  is  needed  for  faculty  mentees  and  student  mentors  to 
build  an  effective  mentoring  relationship  with  each  other.  Mary  and  her  faculty  mentee  Dr.  Johnson 
revealed  the  importance  of  mutual  trust  and  respect  in  their  mentoring.  Dr.  Johnson  said  that  “I  admire  her 
[Mary]  expertise  in  technology  and  wish  to  learn  from  her.  She  showed  great  interest  in  my  course  and 
respects  my  expertise  in  my  subject  matter.  I guess,  we  trust  and  respect  each  other  and  that  helps.” 

Because  mentoring  involves  an  ongoing,  caring  relationship,  a good  mentoring  relationship  can  only  be 
fostered  with  mutual  respect  and  trust  (Janas,  1996).  The  mutual  trust  and  respect  in  the  mentoring 
relationship  allows  ideas,  cooperation,  communication,  and  personal  relationships  to  flourish  (MacArthur, 
1995). 

• Personal  fit.  Personal  fit  between  faculty  mentees  and  their  student  mentors  helps  them  to  establish  a 
meaningful  professional  relationship  with  each  other  when  assigning  partners.  Mentors  and  mentees  who 
have  similar  backgrounds,  common  interests,  and  compatible  personalities  tend  to  have  successful 
mentoring  relationships.  Crystal,  Dr.  Clark  and  Ms.  Davison  who  all  are  language  teachers  were  paired  up. 
They  all  shared  common  interest  in  using  technology  in  language  teaching  and  learning.  Mary  was  matched 
with  Dr.  Johnson  and  Dr.  Taylor  and  they  were  all  interested  in  designing  web  pages  and  online  courses. 
Jane  and  Dr.  Erickson  both  were  interested  in  exploring  PowerPoint. 

• Communication.  Open  dialogue  between  the  mentor  and  mentee  allowing  each  participant  to  express  their 
feelings,  talents,  knowledge  and  expectations  is  essential  for  a successful  mentoring  relationship.  Effective 
mentoring  relationships  occur  when  mentors  and  mentees  clarify  their  roles  and  expectations  for  each  other 
in  mentoring.  Without  good  communication,  student  mentors  and  faculty  mentees  may  have  misconception 
about  mentoring. 

• Mutual  benefit.  Both  the  mentor  and  mentee  should  benefit  from  the  relationship.  Mentees  gain  knowledge, 
skills,  insights,  and  experience.  Mentors  sharpen  skills  and  knowledge,  learn  new  things,  and  reward  from 
the  collaboration  with  the  mentee  (Clemson,  1987).  Both  faculty  mentees  and  student  mentors  must  feel 
they  benefited  from  participating  in  the  mentoring  experience.  They  both  need  to  be  rewarded  from 
participating  in  the  mentoring  relationship.  As  Ms.  Davison  and  Dr.  Clark  stated  that  they  were  "a  step 
further  in  technology”  and  that  their  "comfort  level  is  going  up"  respectively  (Ms.  Davison,  interview;  Dr. 
Clark,  interview).  Student  mentors  indicated  they  refined  their  computer  skills,  became  familiar  with  the 
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department's  context,  gained  professional  knowledge,  and  established  a professional  relationship  with 
faculty  in  the  department  (Jane,  interview;  Mary,  case  report;  Crystal,  interview). 

Research  Question  Two 

What  are  the  roles  of  faculty  mentees  and  what  are  the  roles  of  student  mentors? 

In  the  ISU  student/faculty  mentoring  program,  faculty  mentees  were  motivated  learners  and  active 
participants  in  the  mentoring.  Faculty  mentees  presented  their  learning  needs  and  their  strong  desire  to  learn  by 
setting  learning  goals  and  objectives,  identifying  technology  projects  to  work  on,  and  completing  their  technology 
homework  resulted  from  the  mentoring  sessions.  The  faculty  mentees  really  took  on  the  role  of  the  learner  in  the 
mentoring  experience.  Ms.  Davison  said,  “I  am  definitely  a learner.  Sometimes,  my  mentor  knows  every  question 
and  tutors  me  in  every  aspect.  Sometimes,  she  doesn’t  have  answers,  so  we  learn  together.”  Thus,  the  mentees  play 
roles  of  students  in  the  mentoring  (Clemson,  1987) 

In  the  mentoring  program,  the  roles  of  the  mentors  were  evolving  and  changing  (McArthur,  1995).  Student 
mentors  in  the  ISU  student/faculty  mentoring  program  became  tutors,  coaches,  facilitators,  counselors,  and  learners. 
Faculty  mentees  and  student  mentors  described  the  roles  of  student  mentors  as  a tutor,  a buddy  expert,  a consultant, 
an  advisor,  a guide,  a learning  partner,  a companion,  a friend,  a resource  person,  a coach,  a facilitator,  a collaborator, 
a supporter,  a helper,  and  a good  listener.  The  roles  of  student  mentors  were  summarized  and  discussed  as  follows: 

• The  student  mentors  are  tutors.  Student  mentors  are  the  people  who  can  respond  to  the  needs  of  mentees  by 
providing  instructions  and  feedback  for  faculty  mentees.  They  are  coaches  who  provide  encouragement  and 
direction  for  the  faculty  mentees.  Crystal  taught  Dr.  Clark  Microsoft  Excel  and  Web  page  design  and 
helped  Ms.  Davison  with  Easy  book;  Mary  helped  Dr.  Johnson  learn  WebCT  and  helped  Dr.  Taylor  with 
Imovie  digital  editing;  and  Jan  taught  Dr.  Erickson  how  to  use  PowerPoint.  All  of  the  student  mentors 
demonstrated  their  computer  skills  and  suggested  possible  uses  of  technology  in  the  mentee’s  course.  They 
assessed  how  they  could  assist  faculty  mentees,  what  faculty  had  mastered,  and  which  skills  needed 
reinforcement. 

• The  student  mentors  are  facilitators.  The  mentors  works  as  a resource  people  for  faculty  mentees, 
supporting  and  assisting  them  to  locate  hardware/software  and  to  acquire  desirable  computer  skills.  The 
mentors  facilitate  the  learning  process  with  technology  and  assist  faculty  mentees  to  accomplish  their 
technology  projects.  Crystal  facilitated  Ms.  Davision  to  locate  appropriate  software  for  her  class.  Mary 
helped  Dr.  Johnson  to  learn  WebCT  and  offered  suggestion  to  help  Dr.  Johnson  visualize  his  technology 
project.  Mary  wrote  in  her  student  journal,  “I  feel  like  a facilitator.  The  mentor  role  is  to  help  the  faculty  to 
visualize  what  they  can  do  and  understand  that.  I am  going  to  look  for  information  to  facilitate  the  faculty’s 
learning  process.” 

• The  student  mentors  are  consultants.  The  student  mentors  are  good  listeners  and  consultants.  They  provide 
guidance  as  they  consulted  with  faculty  on  things  they  wanted  to  learn  about  and  complete.  In  the  ISU 
student/faculty  mentoring  program,  student  mentors  listened  to,  probed,  and  provided  suggestions  to  their 
faculty  mentees.  They  were  good  listeners  and  responded  to  their  faculty  mentees  in  a positive  and 
favorable  way.  Jane  offered  advice  and  helped  Dr.  Erickson  to  visualize  his  technology  ideas  for  his  class 
(Jane,  interview).  Crystal  acted  like  a consultant  and  suggested  how  Dr.  Clark  could  use  her  technology 
project  for  her  class  (Dr.  Clark,  Interview).  Likewise,  Mary  helped  Dr.  Johnson  visualize  his  course 
activity  using  WebCT  (Mary,  interview).  Student  mentors  became  counselors  who  advised  faculty  mentees 
on  how  technology  works  and  how  it  can  be  managed. 

• The  student  mentors  are  learners.  Besides  the  traditional  roles  of  mentors,  this  research  study  clearly 
indicated  that  the  student  mentors  learn  technology  together  with  their  faculty  mentees.  Because  technology 
field  is  constantly  changing  and  evolving,  no  one  knows  or  is  expected  to  know  everything  (Brzycki,  Yost 
& Dudt,  2001).  Student  mentors  often  learn  new  computer  software  or  new  skills  together  with  their  faculty 
mentees.  Jane  wrote  in  her  student  journal:  “The  only  thing  I knew  about  PowerPoint  was  to  make  a basic 
slide  when  I began  my  mentoring.  Now  I know  how  to  put  in  animation,  insert  graphics,  and  how  to  put 
slides  together.”  Mary  felt  that  she  “learned  as  much  from  them  and  with  them  [Dr.  Johnson  & Dr.  Taylor]. 
Jane  and  Crystal  also  learned  new  computer  programs  and  technical  skills. 
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Research  Question  Three 

What  is  the  perspective  of  faculty  on  mentoring  regarding  how  and  why  mentoring  works  for  them  to  learn  and  use 
technology? 

Faculty  indicated  that  one-on-one  mentoring  is  an  ideal  situation  for  them  to  learn  about  technology.  They 
reported  that  their  comfort  level  of  using  technology  in  their  teaching  and  research  has  increased.  Dr.  Clark 
expressed  her  favor  for  mentoring:  “I  think  it  [mentoring]  is  a blessing.  I think  the  mentoring  program  is  wonderful. 

I am  a lot  further  in  technology  today  because  of  the  mentoring  program.  I probably  wouldn’t  do  it  otherwise.” 

There  are  six  specific  themes  emerged  from  this  research  study  that  reveals  how  and  why  mentoring  works 
for  faculty  to  learn  and  use  technology. 

• This  mentoring  program  creates  a one-to-one  relationship.  It  produces  a learning  situation  for  faculty, 
where  individualized  instruction  is  designed  to  meet  faculty’s  unique  interest,  technology  experience,  and 
needs.  Data  indicated  participants  were  appreciative  of  being  able  to  work  with  someone  one-on-one. 
Faculty  mentees  learn  at  different  rates  and  bring  a myriad  of  experiences,  needs,  interests,  background, 
and  beliefs  with  them  to  the  learning  and  using  technology  (Bahannon,  2001),.  This  one-on-one  model  is 
helpful  in  adjusing  to  personal  learning  style  and  preferences  and  catering  to  each  faculty  member’s  unique 
interest  and  needs. 

• This  mentoring  program  cultivates  a comfortable  and  nonthreatening  technology  learning  environment. The 
mentoring  relationship  is  an  interpersonal  and  caring  relationship  (Gehrke,  1998).  Dr.  Erickson  said  during 
the  interview,  “I  am  not  in  a group  that  is  better  than  me.  It  intimidates  me  a little  bit.  When  I am  just  in  my 
office  and  someone  is  sitting  next  to  me,  it’s  just  more  comfortable.  No  peer  pressure.” 

• This  mentoring  program  helps  faculty  to  continuously  improve  computer  skills  with  multiple  mentoring 
sessions.  Faculty  repeatedly  commented  that  meeting  with  their  mentors  every  week  helped  them 
continually  learn  about  technology.  Since  the  mentor  was  scheduled  to  meet  with  their  mentee  an  hour 
every  week,  faculty  felt  they  were  “pushed”  to  set  aside  time  to  learn  technology.  Without  meeting 
regularly,  the  faculty  mentees  admitted  that  they  tended  to  “push  technology  down  on  their  working  list” 
(Ms.  Davison,  interview).  The  multiple  and  repeated  mentoring  meeting  sessions  helped  faculty  to  increase 
comfort  level  and  competence  in  using  computers.  This  finding  corresponds  with  the  literature  that  faculty 
members  need  time  to  be  set  aside  to  learn  and  practice  computer  sills  before  they  gain  comfort  level  and 
computer  competency. 

• This  mentoring  program  helps  nurture  a technology  learning  community  for  faculty.  All  faculty  indicated 
that  they  have  use  the  department  technology  center  (Center  of  Technology  in  Teaching  and  Learning  , a 
department  technology  center)  more  often  during  and  after  the  mentoring  program  than  before.  They  came 
to  recognize  “friendly  faces”  down  in  the  technology  center  and  felt  they  were  better  connected  with 
departmental  technology  initiatives.  They  became  more  familiar  with  what  the  department  technology 
center  had  to  offer  and  gradually  began  to  see  themselves  becoming  a part  of  learning  community  where 
everyone  learns  together. 

• This  mentoring  program  gets  extensive  administrative  support  and  encouragement  from  the  Department  of 
Curriculum  and  Instruction.  All  faculty  members  are  encouraged  to  sign  up  to  obtain  assistance  from 
student  mentors.  Without  the  recognition  and  support  from  administrative  level,  faculty  may  not  be 
encouraged  to  put  time  and  effort  to  try  to  integrate  technology  into  their  teaching  (Cobb,  Hensman,  Jones 
& Richards,  1995)  In  addition,  the  department  technology  center  provides  hardware  and  software  and 
personnel  support  for  the  mentoring  program.  Because  inadequate  or  obsolete  equipment  and  limited 
availability  of  equipment  would  block  faculty  from  their  usage  of  technology  (Cuban,  1993),  the  CTLT  acts 
an  important  in  the  student/faculty  mentoring  program  to  ensure  the  accessibility  of  hardware  and  software 
by  faculty  mentees.  Along  with  the  good  accessibility  of  technology  resources,  the  CTLT  provides  good 
personnel  support  to  the  student/faculty  mentoring  program.  Student  mentors  and  faculty  mentees  directed 
their  questions  to  the  technical  experts  in  the  center  and  department  and  received  help  to  solve  their 
technical  problems.  Jane  commented,  'T  can't  imagine  what  my  mentoring  experience  would  be  if  there's 
nobody  in  the  center  [CTLT]  that  can  help  me."  (Jane,  interview). 

Research  Question  Four 

What  is  the  perspective  of  student  mentors  on  mentoring  program  regarding  assisting  faculty  learning  and  using 
technology? 

Student  mentors  reported  that  mentoring  was  an  excellent  way  to  help  faculty  learn  to  use  technology.  They 
agreed  that  this  mentoring  program  was  very  helpful  and  effective  to  assist  technology  integration  into  their  courses. 
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There  are  four  themes  that  emerged  from  student  mentors’  perspective  about  how  and  why  this  mentoring  program 
works: 

• Individualized  instruction  to  faculty  mentees  is  critical  to  help  faculty  learn  and  use  technology.  Like 
faculty,  student  mentors  feel  that  one-on-one  nature  of  mentoring  relationship  creates  the  best  situation  to 
cater  to  individual  needs  of  faculty  mentees. 

• Providing  a private  and  comfortable  learning  environment  enables  faculty  to  learn  technology  in  a non- 
threatening way.  According  the  student  mentors,  this  is  an  advantage  of  mentoring  over  traditional 
workshop  models.  This  mentoring  model  provided  a private  and  secure  environment  for  faculty  mentees  to 
learn  technology  without  worrying  about  asking  “stupid”  questions  in  front  of  peers. 

• This  mentoring  program  is  appealing  and  effective  because  it  focused  on  assist  faculty  mentees  to  learn  and 
use  technology  in  their  “specific’  courses.  It  gave  them  the  opportunity  to  explore  technology  activities  in 
their  own  courses.  Instead  of  learning  technical  skills  superficially,  faculty  mentees  are  actually 
designing/developing  technology  projects  for  their  own  courses. 

• This  mentoring  program  allows  student  mentors  and  faculty  mentees  to  receive  support  from  C I 610  and 
the  department  technology  center.  Faculty  mentees  and  student  mentors  received  help  from  the  technology 
center’s  staff.  C I 610  course  provided  student  mentors  with  a secure  place  to  share,  discuss,  and  get  advice 
about  their  individual  mentoring  experience.  This  kind  of  support  is  essential  to  the  success  of  this 
mentoring  program. 

Conclusion 

In  conclusion,  this  research  investigated  how  and  why  the  student/faculty  mentoring  program  in  the 
Department  of  Curriculum  and  Instruction  at  Iowa  State  University  assists  teacher  education  faculty  with  technology 
integration.  The  research  explored  characteristics  of  effective  mentoring  relationships,  the  roles  of  student  mentors 
and  faculty  mentees  in  the  student/faculty  mentoring  program,  and  the  perspectives  from  the  faculty  mentees  and 
student  mentors  about  mentoring. 

Research  results  indicated  that  characteristics  of  effective  mentoring  relationships  of  ISU  student/faculty 
mentoring  program  are:  time,  mutual  trust  and  respect,  personal  fit,  communication,  and  mutual  benefit.  In  this 
student/faculty  mentoring  program,  faculty  mentees  were  active  learners  and  motivated  participants.  Student 
mentors  were  tutors,  facilitators,  counselors,  and  learners.  This  student/faculty  mentoring  program  was  effective 
because  of:  the  one-on-one  individualized  instruction;  the  non-threatening  technology  learning  environment;  the 
continuous  improvement  of  computer  skills  with  multiple  mentoring  meeting  sessions;  the  fact  that  faculty  allotted 
one  hour  a week  to  work  on  technology,  a nurturing  technology  learning  community;  and  administrative  support  and 
encouragement. 

This  research  study  deepens  the  understanding  of  how  and  why  using  student  mentors  is  a promising 
strategy  to  assist  teacher  education  faculty  with  technology  use  and  integration.  The  results  providevaluable 
information  and  experience  about  how  to  design  an  effective  student/faculty  mentoring  program.  Teacher  education 
programs  interested  in  using  such  an  approach  should  work  toward  making  sure  these  characteristics  are  present  and 
should  clarify  the  roles  of  student  mentors  and  faculty  mentees  to  design  an  effective  student/faculty  mentoring 
model. 
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Abstract 

Feedback  is  considered  an  important  instructional  variable  in  improving  students  ’ achievement.  Although 
several  studies  have  indicated  that  teacher  feedback  is  a powerful  variable  in  terms  of  enhancing  students’  learning, 
the  issue  of  peer  feedback  has  not  been  well  explored.  The  present  study  examined  the  effects  of  various  different  of 
peer  feedback  on  learner  achievement  and  motivation.  Within  three  sections  in  a Technology  in  Secondary  Teaching 
course  (n  = 35),  the  three  feedback  treatments  were  assigned  to  intact  groups.  A two- factor  repeated  measures 
ANOVA  was  used  to  compare  achievement  and  attitude  across  time  and  across  the  treatments.  Although  each  group 
showed  improvement  upon  receiving  feedback,  a significant  difference  for  treatments  was  not  found.  Due  to  the 
positive  impact  of  peer  feedback  for  both  raters  and  ratees,  suggestions  for  refining  this  area  of  research  are 
discussed. 

Background 

Feedback  has  been  found  to  be  an  important  instructional  variable  in  improving  student  achievement. 
Bangert -Drowns,  Kulik  and  Morgan  (as  cited  in  Mory,  1996)  indicated  that  feedback  is  incorporated  in  both 
behavioral  and  cognitive  learning  paradigms  and  is  an  essential  element  of  theories  of  learning  and  instruction. 
Clariana,  Ross,  and  Morrision  (1991)  noted  that  while  the  benefit  of  feedback  in  general  is  taken  for  granted, 
uncertainty  still  exists  as  to  how  to  select  and  optimize  the  uses  of  different  forms  of  feedback  depending  on 
characteristics  of  students  and  the  learning  situation.  This  notion  was  supported  by  Clark  and  Dwyer  (1998)  who 
stated  that  knowledge  of  response  feedback  enhanced  performances  when  the  task  involved  discrimination  or  rote 
learning. 

Although  several  studies  (Silverman,  1992;  Elwell  & Tiberio,  1994)  have  indicated  that  teacher  feedback  is 
a powerful  variable  in  terms  of  enhancing  students’  learning  (as  cited  in  Burnett,  2001),  the  issue  of  peer  feedback 
has  not  been  well  explored. 

Peer  feedback  refers  to  when  students  learn  from  each  other  by  exchanging  and  comparing  their  particular 
strategies  or  ideas  on  instructional  activities.  Peer  feedback  may  be  a way  to  increase  student  learning  and 
motivation.  The  present  study  investigates  effects  of  varying  levels  of  peer  feedback  among  students. 

Research  Questions 

The  following  research  questions  were  investigated: 

1 . Do  different  types  of  peer  feedback  have  different  effects  on  learner  achievement? 

2.  What  type  of  peer  feedback  can  best  stimulate  students’  motivation  to  learn? 

Method 

Subjects 

The  subjects  were  35  undergraduate  students  in  three  different  sections  of  a Technology  in  Secondary 
Teaching  course.  All  subjects  are  pre-service  teachers  and  this  technical  class  is  required  of  them.  All  of  them  are 
undergraduate  students  from  sophomore  to  senior.  There  were  13  males  and  22  female  with  a mean  age  of  22.4. 

Each  section  was  assigned  to  one  of  the  three  levels  of  feedback  treatment  (CE-CE  Eeedback  group,  EE-CE 
Eeedback  group,  or  EE  -EE  Eeedback  group).  The  Checklist  Eeedback  (CE)  is  a simple  level  of  feedback,  which 
provides  check  marks  on  the  rubric  to  tell  subjects  which  level  they  reached  and  their  final  scores.  The  Elaborative 
Eeedback  (EE)  is  feedback  of  a higher  level.  It  not  only  contains  CE  information,  but  also  provides  reasons  why 
students  received  certain  scores  for  each  category  and  relevant  revision  suggestions. 
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Materials 

As  required  for  their  final  projects,  subjects  created  a personal  website,  which  was  an  electronic  portfolio  to 
show  their  work  and  prepare  them  for  their  job  hunt.  This  web  site  included  their  (1)  Basic  Information:  education, 
work  history  and  accomplishments,  etc.  (2)  Teaching  philosophy  and  (3)  Examples  of  work  they  have  done  in 
preparation  for  a teaching  position. 

A rubric  was  given  to  the  subjects  before  they  started  to  do  the  project  to  indicate  how  the  assignment 
would  be  evaluated.  The  grading  criteria  were  divided  into  five  major  categories:  content,  presentation,  navigation, 
links,  and  mechanics.  The  total  possible  points  are  34. 

Subjects  completed  a 6-item  inventory  designed  for  this  study  to  indicate  students’  attitudes  toward 
feedback  before  the  treatment.  The  first  three  items  were  designed  to  measure  students’  beliefs;  the  other  three 
address  comfort.  These  questions  include: 

(1) 1  believe  receiving  feedback  from  my  instructor  will  help  me  improve  my  work. 

(2)  I believe  receiving  feedback  from  my  peers  will  help  me  improve  my  work. 

(3)  I believe  providing  feedback  to  other  people  would  help  me  learn  more. 

(4)  I feel  comfortable  receiving  feedback  from  my  instructor. 

(5)  I feel  comfortable  receiving  feedback  from  my  peers. 

(6)  I feel  comfortable  providing  feedback  to  my  peers. 

Response  options  to  these  items  range  from  strongly  agree  to  strongly  disagree  on  a five-point  Likert  scale  (5  = 
strongly  agree  to  1 = strongly  disagree). 

After  the  feedback  treatment,  subjects  completed  the  same  6-item  inventory  and  four  additional  open-ended 
questions  as  listed  below: 

(1)  What  did  you  like  most  about  this  approach  on  providing  feedback  to  peers? 

(2)  What  did  you  like  most  about  this  approach  on  receiving  feedback  from  peers? 

(3)  What  do  you  like  least  about  this  approach  on  providing  feedback  to  peers? 

(4)  What  do  you  like  least  about  this  approach  on  receiving  feedback  from  peers? 

Treatment 

Each  section  was  assigned  to  one  of  the  three  levels  of  feedback  treatment  (CE-CE,  EE-CE  or  EE-EE).  The 
researcher  randomly  assigned  the  intact  groups  to  the  different  levels  of  treatment.  The  grading  criteria  were  given 
to  each  student  in  advance  and  students  provided  their  feedback  in  a CE  form  or  in  an  EE  form. 

(1)  CE-CE  Treatment  («  = 17).  Subjects  provided  CE  to  peers . The  instructor  organized  their  feedback  and  then 
returned  the  final  results  as  CE  to  subjects . 

(2)  EF-CE  Treatment  (n  = 1 1).  Subjects  provided  EE  to  peers.  The  instructor  organized  their  feedback  and  then  only 
provided  the  final  results  as  CE  to  subjects. 

(3)  EE  - EE  Treatment  («  = 7).  Subjects  provided  EE  to  peers . The  instructor  organized  their  feedback  and  then 
returned  the  final  results  as  EF  to  subjects  . 

The  CF-CF  group  provided  CE  to  their  peers;  the  EF  -CE  and  EF -EF  group  provided  EF  to  the  peers  in  their  class. 

All  students  taking  the  class  were  required  to  complete  the  website  project  regardless  of  whether  or  not  they  were 
involved  in  the  research  study. 

Procedure 

Students  who  enrolled  in  the  Technology  in  Secondary  Teaching  course  were  selected  to  participate  in  the 
study.  Subjects  were  contacted  initially  by  announcing  the  project  orally  in  the  beginning  of  the  class.  Students  were 
asked  to  sign  a letter  of  consent  if  they  wished  to  participate  and  asked  to  complete  the  Students  Learning 
Motivation  (SLM)  Survey  questionnaires.  General  demographic  information  (i.e..  Last  6 digits  of  subjects’  social 
security  number,  major,  gender,  age  and  class  section  number)  was  solicited  on  the  questionnaires,  but  no  other 
identification  of  respondents  was  collected.  The  researcher  informed  subjects  that  the  purpose  of  this  study  was  to 
examine  the  factors  that  affect  learning.  Subjects  knew  that  they  would  be  asked  to  participate  in  some  learning 
activities,  such  as  giving  feedback  to  peers,  but  they  did  not  know  which  treatment  group  they  were  in  or  the 
researcher’s  expectations. 

Subjects  went  through  the  same  instructional  content  over  a three-week  period  that  was  taught  by  the  same 
instructor,  and  then  worked  on  the  website  assignment  for  one  week. 

After  receiving  the  assignment  via  E-mail,  the  instructor  graded  each  assignment  and  recorded  the  first 
score  based  on  the  rubric.  Then,  the  instructor  uploaded  each  student’s  assignment  to  the  class  website  and  allowed 
subjects  to  view  their  peers’  assignments  online.  Students  received  the  first  scores  and  also  were  informed  that  only 
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the  revised  scores  would  be  recorded  as  the  projects  final  scores.  Students  also  received  bonus  credit  for  providing 
feedback  according  to  the  feedback  quality. 

After  subjects  saved  their  feedback  to  floppy  disks  and  gave  them  to  the  instructor,  the  feedback  was 
organized,  put  into  the  proper  format,  and  provided  to  the  website  projects’  creators.  Then,  the  instructor  asked 
subjects  to  revise  their  projects  according  to  the  feedback  they  received  and  send  the  revised  projects  back  within 
one  week.  The  instructor  evaluated  the  revised  projects  and  gave  a final  score  to  each  student.  After  the  students 
turned  in  the  revised  projects,  the  subjects  completed  the  SLM  Survey. 


Criterion  Measures 

A two-factor  repeated  measures  ANOVA  was  used  to  analyze  the  achievement  and  attitude  of  individuals , 
across  the  CF-CF,  EF-CF  and  EF-EF  treatments  and  across  the  impact  of  the  feedback.  After  finding  a significant 
main  effect,  the  researcher  ran  post-hoc  ANOVAS  to  determine  the  nature  of  the  interaction. 

The  achievement  scores  were  collected  from  subjects’  original  and  revised  website  projects  according  to 
the  rubric.  The  attitude  scores  were  collected  from  subjects’  pre  and  post  SLM  Surveys.  A significance  level  of  .05 
was  used  to  run  the  analyses. 

Result 

The  reported  results  for  this  investigation  are  separated  into  two  sections:  the  effects  on  learner 
achievement  with  different  types  of  peer  feedback  and  the  relationship  between  peer  feedback  and  students’  learning 
motivation 
Achievement 


Means  and  standard  deviations  of  the  responses  for  the  first  and  revised  projects  of  treatment  groups  are 
reported  in  Table  1. 

Table  1 Mean  and  Standard  Deviation  for  the  Treatment  Groups 


Treatment  Groups 

M 

SD 

n 

Eirst  Project 

CE-CE 

26.94 

4.67 

\i 

EE-CE 

16.91 

7.02 

1 

EE-EE 

23.64 

2.61 

11 

Total 

23.13 

6.77 

35 

Revised  Project 

CE-CE 

29.21 

3.91 

17 

EE-CE 

27.09 

6.70 

7 

EE-EE 

31.00 

4.37 

11 

Total 

28.90 

5.08 

35 

Note.  The  total  possible  point  for  the  website  project  is  34. 


A two-factor  repeated  measures  ANOVA  revealed  the  mean  of  the  revised  projects  was  significantly  higher 
than  the  mean  of  the  first  projects  across  the  three  treatments,  F = 67.86,  p < .01.  As  expected,  the  students  had 
shown  significant  improvement  of  their  revised  projects  (see  Figure  1). 

FIGURE  1 . The  Achievement  for  Three  Treatment  Groups 


Dreamweaver  Projects 
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Table  2.  Multivariate  Tests 


Effect 

F 

P 

Project 

67.86 

<01 

Treatments 

6.09 

<01 

Project  * Treatments 

11.15 

<01 

There  was  an  overall  significant  interaction  between  the  repeated  factor  and  the  treatments,  F=  11.15,  p< 
.01  (see  Table  2).  Follow-up  post-hoc  ANOVAS  revealed  that  there  was  no  significant  interaction  in  EF-EF  vs.  EF- 
CF,  F = 1.07,  p = .32.  However,  there  is  a significant  disordinal  interaction  showed  in  EF-EF  vs.  CF-CF,  F = 12.63, 
p < .01,  and  a significant  ordinal  interaction  in  EF -CF  vs.  CF-CF,  F = 20.47,  p < .01  (see  Figurel  and  Table  3). 


Table  3.  Interaction  between  Projects  and  Treatments 


Treatment  Groups 

F 

P 

EE-EE  vs.  EE-CE 

1.07 

.32 

EE-EEvs.  CE-CE 

12.63 

<.01 

EE-CE  vs.  CE-CE 

20.47 

<.01 

ANOVA  was  conducted  for  comparing  scores  of  the  first  projects  (see  Table  4)  to  examine  the  pre- 
treatment difference.  For  the  first  score,  EF-EF  was  significantly  higher  than  EF-CF  (p  = .03),  EF-EF  was  not 
significantly  different  with  CF-CF,  and  EF-CF  was  significantly  lower  than  CF-CF  (p  < .01)  (see  Table  4 and  Figure 
1).  Because  these  two  groups  (EF-EF  vs.  CF-CF  and  EF  -CF  vs.  CF-CF)  already  had  a pre-existing  difference  before 
the  treatment,  it  was  hard  to  claim  the  interaction  came  from  the  treatment  not  from  their  original  difference. 


Table  4.  Projects’  First  Scores  Comparison 


Treatment  Groups 

P 

Projects’  Eirst  Scores 

EE-EE  vs.  EE-CE 

.03* 

EE-EEvs.  CE-CE 

.35 

EE-CE  vs.  CE-CE 

< .01* 

p < .05 


Students  Learning  Motivation 

The  first  three  questions  addressed  student  beliefs;  the  other  three  addressed  comfort.  Questions  one  and 
four  of  the  survey  dealt  with  receiving  feedback  from  instructors  (Q 1 : 1 believe  receiving  feedback  from  my 
instmctor  will  help  me  improve  my  work;  Q4:  I feel  comfortable  receiving  feedback  from  my  instructor).  Questions 
two  and  five  looked  at  the  benefit  and  comfort  levels  of  receiving  feedback  from  peers  (Q2: 1 believe  receiving 
feedback  from  my  peers  will  help  me  improve  my  work;  Q5: 1 feel  comfortable  receiving  feedback  from  my  peers). 
Questions  three  and  six  regarded  providing  feedback  to  peers  (Q3: 1 believe  providing  feedback  to  other  people 
would  help  me  learn  more;  Q6: 1 feel  comfortable  providing  feedback  to  my  peers.). 

The  researchers  first  ran  a Cronbach’s  alpha  on  the  set  of  six  SLM  survey  items  to  examine  the 
internal  reliability.  The  Cronbach’s  alphas  reliability  coefficient  for  the  pre  and  post  SLM  surveys  were  .79  and  .87. 

A follow-up  two-factor  repeated  measures  ANQVA  to  compare  the  attitudes  across  time  and 
across  the  treatments.  The  scores  were  collected  from  subjects’  pre  and  post  SLM  surveys  based  on  the  means  of 
the  six  survey  items  (see  Table  4).  There  was  no  significant  interaction  between  the  administrations  and  treatments, 
p = .24. 


Table  5.  Mean  and  Standard  Deviation  for  Pre  and  Post  SLM  Surveys 


Survey 

Pre 

Post 

Questions 

M 

SD 

M 

SD 

Ql 

I believe  receiving  feedback  from  my  instructor  will  help  me  improve  my  work. 

4.51 

.92 

4.51 

.82 

Q2 

I believe  receiving  feedback  from  my  peers  will  help  me  improve  my  work. 

3.86 

.88 

3.89 

1.16 

Q3 

I believe  providing  feedback  to  other  people  would  help  me  learn  more. 

3.86 

.85 

3.91 

1.01 

Q4 

I feel  comfortable  receiving  feedback  from  my  instructor. 

4.51 

.66 

4.34 

.87 

Q5 

I feel  comfortable  receiving  feedback  from  my  peers. 

3.80 

.93 

3.97 

1.07 

Q6 

I feel  comfortable  providing  feedback  to  my  peers. 

3.94 

.94 

3.94 

1.03 

Note.  Reponses  ranged  from  1 (Strongly  Disagree)  to  5 (Strongly  Agree). 
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Students  feel  at  least  as  comfortable  receiving  feedback  fro  m peers  as  before  the  treatment  (M  =3.80  and  M 

= 3.97). 

With  regard  to  the  open-ended  instrument,  the  first  two  questions  address  student  likes  about  this  approach; 
the  other  two  address  student  dislikes  about  this  approach.  Questions  one  and  three  dealt  with  providing  feedback  to 
peers  (Ql:  What  did  you  like  most  about  this  approach  on  providing  feedback  to  peers;  Q3:  What  did  you  like  least 
about  this  approach  on  providing  feedback  to  peers).  Questions  two  and  four  regarded  receiving  feedback  frompeers 
(Q2:  What  did  you  like  most  about  this  approach  on  receiving  feedback  from  peers;  Q4:  What  did  you  like  least 
about  this  approach  on  receiving  feedback  from  peers) 

When  providing  feedback  to  peers,  eight  students  in  the  CF-CF  («  = 17)  group  reported  it  was  way  too  time 
consuming  for  them  at  the  end  of  the  semester  and  a lot  of  extra  work.  Four  students  reported  that  this  approach  took 
way  too  long  and  was  tedious.  Five  students  in  EF  -CF  group  («  = 1 1)  also  thought  providing  feedback  to  peers  was  a 
time  consuming  process.  Two  students  felt  uncomfortable  and  did  not  feel  like  it  was  their  place  to  give  peers 
feedback.  However,  seven  students  had  a positive  attitude  for  this  approach.  They  reported  it  enabled  them  to  see 
how  much  they  could  improve  their  own  project  by  seeing  peers  ’ projects  that  were  in  depth.  All  of  the  students  in 
the  EF-EF  group  («  = 7)  reported  there  was  nothing  they  disliked  about  this  approach  on  providing  feedback  to 
peers. 

When  receiving  feedback  from  peers,  the  two  students  in  CF-CF  {n  = \1)  group  reported  that  CF  does  not 
really  tell  them  what  they  need  to  improve  or  the  reasons  they  received  a low/high  grade.  In  the  EF-CF  group  (n  = 
11),  five  students  disliked  this  approach  because  they  did  not  get  peers  ’ comments;  only  the  scores.  In  the  EF-EF 
group  (m  = 7),  four  students  reported  there  was  nothing  they  did  not  like  about  receiving  feedback  from  others  For 
example,  “I  was  able  to  see  different  peoples  opinions  on  what  I could  do  better.”  and  “Getting  to  see  the  different 
things  people  suggested  I improve  upon”. 

Most  of  students  had  a positive  attitude  towards  this  peer  approach.  They  reported  it  enabled  them  to  see 
how  much  they  could  improve  their  own  project  by  seeing  their  peers  ’ projects  that  were  in  depth. 

Discussion 

The  results  of  this  exploratory  study  suggest  three  findings  regarding  peer  feedback  in  student  work  and 
provoke  a number  of  questions  for  further  research.  First,  feedback  was  useful  for  improving  students’  scores. 
However,  the  specific  effects  the  treatments  were  confounded  in  this  study.  Second,  students'  perceptions  of  what 
they  learned  by  viewing  peers’  projects  were  generally  positive  although  some  had  reservations  about  its  potential  to 
improve  their  own  work.  Most  students  believed  that  the  feedback  they  received  after  their  projects  were  evaluated 
by  a peer  was  helpful;  however,  a few  had  a more  negative  view  of  the  experience.  Third,  students'  perceptions  of 
the  comfort  level  they  experienced  providing  and  receiving  feedback  form  peers  were  generally  positive. 

Peer  Feedback  Level 

The  EF-EF  group  that  provided  and  received  an  Elaborative  Eeedback  (EE)  form  their  peers  had  higher 
achievement  than  the  CE-CE  group  that  provided  and  received  Checklist  Eeedback  (CE)  form  to  their  peers. 

Because  the  CE  provides  only  check  marks  on  the  rubric  and  their  final  scores,  the  CE-CE  students  reported 
that  CE  does  not  really  tell  them  what  they  need  to  improve  or  why  they  received  a low/high  grade.  The  students  in 
the  EE-EE  treatment  were  able  to  see  different  peoples  opinions  on  what  they  could  do  better  and  get  to  see  the 
different  things  people  suggested  they  improve  upon.  They  can  fix  the  projects  and  saw  things  wrong  with  it  after 
peers  critiqued  their  projects.  However,  this  result  is  reported  guardedly  due  to  other  is  sues  with  the  study. 

Apossible  confoundis  suggested  by  Dominick’s  (1997)  findingofno  significant  difference  between  the 
feedback  group  (participants  who  give  and  receive  behavior  peer  feedback)  and  exposure  group  (participants  who 
give,  but  do  not  receive  peer  feedback).  In  other  words,  the  benefits  of  giving  elaborative  feedback  may  outweigh 
the  limitation  of  receiving  less  feedback. 

(1)  Learning  from  viewing  peers’  projects.  This  finding  centered  on  the  learning  involved  in  viewing  peers’ 
projects  and  providing  feedback  to  peers.  Students  showed  improvement  in  their  revised  projects  because  they 
received  the  new  ideas  the  peers  had,  which  they  applied  in  their  projects.  Peers  reflected  on  how  they  might 
improve  a particular  part  when  they  observed  or  noted  differences  in  style.  They  then  were  able  to  rework  their 
problem  areas.  When  students  provided  feedback  to  peers,  they  also  noticed  what  the  weakness  and  strengths  for 
their  own  projects  were,  no  matter  which  treatment  groups  they  were  in.  It  also  helped  them  to  evaluate  their  own 
projects.  These  comparisons  may  have  been  all  the  more  relevant  since  they  had  both  experienced  essentially  the 
same  educational  program  and  had  a shared  knowledge  base  and  level  of  discourse.  Since  their  next  desired  step  is  a 
teaching  position,  there  is  a strong  inclination  to  try  to  improve  their  electronic  portfolio  project  after  viewing 
others’  projects. 


281 


(2)  Students’  grading  attitude.  Some  students  did  not  learn  from  this  peer  approach  because  they  felt 
uncomfortable.  It  may  be  some  peers  were  a little  too  nice  in  providing  feedback  in  order  not  to  hurt  someone’s 
feelings  instead  of  being  honest  regarding  what  they  thought  could  really  be  improved.  Another  explanation  for 
these  less  positive  perceptions  is  that  they  felt  their  peers  might  not  have  taken  it  seriously  and  did  not  provide 
honest  feedback.  Not  all  students  engaged  in  the  process.  Therefore,  these  students  did  not  engage  in  the  opportunity 
to  reflect  on  the  experiences  and  gain  insight  from  each  other.  Some  students  felt  confused  because  they  received 
different  feedback  in  the  same  category,  and  they  did  not  know  which  direction  should  they  go. 

(3)  Weakness  of  the  design  and  rubric.  In  the  Nonequivalent  Control-Group  Design  (NEGD),  because  it  is 
often  likely  that  the  groups  are  not  equivalent,  as  a result,  the  groups  may  be  different  prior  to  the  study.  That  is,  the 
NEGD  is  especially  susceptible  to  the  internal  validity  threat  of  selection.  Any  prior  differences  between  the  groups 
may  affect  the  outcome  of  the  study.  The  main  threat  to  the  internal  validity  of  a NEGD  experiment  is  the  possibility 
that  group  differences  on  the  posttest  are  due  to  pre-existing  group  differences  rather  than  a treatment  effect.  In  this 
case,  because  in  EF-EF  vs.  EF-CF  and  EF-CF  vs.  CF-CF  comparison  already  had  significant  differences  on  their 
first  scores,  even  though  EF-CF  vs.  CF-CF  comparison  had  significant  interaction,  this  may  be  due  to  pre-existing 
group  differences  rather  than  a treatment  effect. 

There  is  also  a doubt  about  the  rubric  design.  In  EF-CF  treatment,  subjects  received  CF  from  peers.  In  EF-  EF 
treatment,  subjects  received  EF  from  peers.  Although  it  seems  there  are  two  types  of  feedback  on  the  receiving  part, 
the  rubric  contained  elaborate  narrative  even  in  the  CF  treatment.  Because  the  rubric  clarified  why  students  received 
certain  scores  for  each  category,  even  the  subjects  in  the  EF-CF  treatment  who  received  CF  received  similar 
information  as  the  subjects  in  the  EF-EF  treatment. 

Students’  Learning  Motivation 

A possible  reason  why  there  was  no  significant  difference  in  comparing  the  attitudes  across  time  and  across 
the  treatments  is  the  ceiling  effect.  The  means  on  the  pre  survey  were  above  3.8  on  a five-point  Likert  scale  (5  = 
strongly  agree  to  1 = strongly  disagree),  leaving  relatively  little  room  for  improvement  on  the  post  survey. 

Student  teachers  had  a positive  perception  of  their  comfort  level  and  the  instructional  value  of  feedback 
received  from  two  types  of  evaluators:  instructor  and  peers.  More  specifically,  students  had  the  most  positive 
perceptions  of  a reciprocal  peer  evaluation  process  that  asked  them  to  provide  and  receive  elaborative  feedback 
(EF).  The  ratings  indicate  that  student  teachers  feel  most  comfortable  in  receiving  feedback  from  their  instmctor. 

Students  have  the  same  comfort  level  on  providing  feedback  to  peers  as  before  the  treatment  (M  = 3.94).  Since 
the  sample  size  in  the  CF-CF  group  was  the  largest  among  the  three  treatment  groups,  and  students  needed  to  view 
through  their  peers’  projects  (m  = 17)  and  then  gave  peers  the  checklist  feedback,  most  students  reported  it  was  way 
too  time  consuming  for  them  at  the  end  of  the  semester  and  a lot  of  extra  work. 

Since  the  students  just  needed  to  provide  check  marks,  students  could  complete  the  feedback  process  with 
little  concern  as  to  its  value.  However,  some  students  were  not  satisfied  with  their  grades.  They  believed  that  the 
grader  did  not  fully  understand  the  rubric  grading  system  or  its  instructions  because  he  or  she  provided  ambiguous 
feedback  to  others.  In  other  words,  they  did  not  think  their  peers  really  cared  or  took  their  time  to  actually  grade 
their  project. 

The  researchers  believe  that  peer  feedback  was  usually  welcomed,  particularly  when  it  was  constructive  or 
prompted  new  insights.  They  appreciated  honest  and  thoughtful  suggestions  and  alternative  ideas.  On  many 
occasions,  students  reported  that  the  questions  and  comments  of  their  peers  prompted  them  to  offer  an  explanation 
of  their  projects. 

It  appears  that  some  students  did  not  learn  from  each  other  for  a variety  of  reasons.  When  students  showed 
little  interest  in  their  peers’  project  but  rather  viewed  the  process  as  a requirement  to  complete,  they  reported 
learning  little.  They  provided  feedback  casually  and  had  less  motivation  to  revise  their  own  projects.  It  is  possible 
that  students  who  differed  in  learning  motivation,  experience,  or  personality  benefited  less  from  their  learning 
experience. 

Suggestions  for  the  Future  Study 

Because  each  treatment  group  showed  improvement,  further  research  might  use  a more  traditional  control 
group.  The  researcher  could  then  investigate  the  interactive  effects  between  varying  levels  of  feedback  and  no 
feedback  (control  group).  Because  the  subjects  might  learn  from  viewing  and  providing  feedback  to  peers  without 
receiving  feedback  from  peers,  future  research  might  examine  the  interactive  effects  on  providing  peer  feedback 
procedure  only. 

The  main  threat  to  the  internal  validity  of  a nonequivalent  control-group  experiment  is  the  possibility  that 
group  differences  on  the  posttest  are  due  to  pre-existing  group  differences  rather  than  a treatment  effect.  The  future 
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research  should  reduce  the  effects  of  initial  group  differences  by  applying  treatment  to  intact  groups  who  have 
similar  performance  before  the  treatment. 

Because  this  research  asked  subjects  to  provide  feedback  to  every  individual  project,  subjects  felt  it  was  a 
tedious  process  and  even  lost  patience  for  providing  serious  feedback.  The  future  study  might  let  students  work  as 
small  groups  to  discuss  the  strengths  and  weaknesses  about  peers’  project,  and  the  students  can  reduce  workload  by 
cooperative  learning.  The  feedback  provided  by  students  also  should  be  counted  as  part  as  to  their  project  grades  to 
adjust  their  attitudes  in  providing  feedback. 
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The  use  of  the  Internet  in  distance  education  has  increased  at  a tremendous  pace.  By  1997  forty  percent  of 
all  higher  education  institutions  in  the  United  States  offered  some  type  of  online  distance  learning  course  (Council 
for  Higher  Education  Accreditation,  1999).  In  March  of  2000,  the  Los  Angeles  Time  reported  that  one  in  three  U.S. 
colleges  offered  an  accredited  degree  online  (Los  Angles  Times,  2000).  Many  education  scholars  agree  that 
applications  of  web-based  instruction  have  developed  at  a faster  pace  than  any  other  development  in  the  history  of 
higher  education  (Twigg  & Oblinger,  1996;  Thrall,  1999;  Gladieux  & Swail,  1999). 

The  Development  of  Virtual  Institutions 

The  use  of  the  Internet  to  teach  distance  education  courses  has  transcended  from  a type  of  distance 
education  course  to  new  types  of  institutions  that  provide  greater  access  to  higher  education.  Institutions  of  higher 
education  and  state  legislatures  have  created  virtual  collaborations  as  an  opportunity  to  extend  the  benefits  of  higher 
education  beyond  its  traditional  boundaries.  Although  there  have  been  a number  of  consortiums  developed  that 
could  be  titled  virtual  universities.  Western  Governors  University  was  the  first  virtual  university  in  the  United  States 
to  attract  attention  from  the  mass  media  (WGU,  2000).  By  March  of  1999,  the  State  Higher  Education  Executive 
Officers  (SHEEO)  published  a report  that  stated  that  thirty-three  state  or  regional  consortia  had  developed  virtual 
universities  (Council  of  Higher  Education  Accreditation,  1999). 

Due  to  the  newness  of  the  virtual  university,  there  is  little  research  in  this  area,  particularly  research  with  a 
focus  on  policy  development.  In  an  article  published  in  the  Technoloev  Source  in  March  of  1999,  Trevor  Thrall 
argued  that  Virtual  Universities  have  bypassed  some  of  the  traditional  decision-making  processes  due  to  their  lack  of 
a brick  and  mortar  mindset.  A great  deal  of  public  funding  has  been  directed  towards  virtual  universities  without  a 
comprehensive  examination  of  the  policy  characteristics  of  these  institutions.  This  article  reports  on  a policy 
analysis  that  examined  the  development  process  of  public  virtual  universities.  The  study  first  classified  existing 
public  virtual  universities  in  the  United  States  using  Wolf  and  Johnstone’s  (1999)  taxonomy,  then,  using  a content 
analysis  approach,  examined  policy  development  for  institutions  falling  under  selected  classifications.  A focus  on 
public  virtual  universities  was  chosen  due  to  the  fact  that  private  institutions  do  not  have  the  same  government 
mandated  policy  requirements  as  public  institutions.  Eighteen  public  virtual  institutions  were  identified  as  operating 
at  the  time  of  this  study  (2000).  One  additional  public  virtual  institution,  which  had  recently  ceased  operation  (in 
1999),  was  also  included. 

Wolf  and  Johnstone’s  Institutional  Taxonomy 

In  May  of  1999  David  B.  Wolf  and  Sally  M.  Johnstone  published  an  article  in  Change  entitled  Cleaning  up 
the  Language  of  Electronically  Delivered  Academic  Programs.  They  argued  that  the  language  used  to  refer  to  new 
forms  of  electronically  delivered  programs  was  disorganized  and  chaotic,  pointing  to  the  fact  that  there  are  several 
meanings  for  the  same  phrase.  The  authors  cited  the  use  of  the  phrase  ‘virtual  university’  as  an  example.  The 
authors  noted,  “We  sometimes  use  one  term  to  refer  to  very  different  types  of  arrangements’’  (Wolf  & Johnstone, 
1999,  p.  1).  In  order  to  provide  clarification  to  this  issue  the  authors  “developed  a taxonomy  to  help  make  the 
variety  of  organizational  arrangements  used  to  administer  electronic  offerings  more  explicit’’  (Wolf  & Johnstone, 
1999,  p.2).  The  institutional  taxonomy  was  created  to  take  advantage  of  conventional  terms  that  align  with  accepted 
existing  structures  of  higher  education.  The  Institutional  Taxonomy  is  as  follows: 

Virtual  University  / College  - academic  degree  granting  with  no  campus. 

Virtual  University  Consortium-  no  degree  granted  but  accredited  academic  institutions  are  linked  online 
and  supply  centralized  or  coordinated  services  to  students  with  mutual  articulation  among  consortium 
members. 

Academic  Services  Consortium-  no  degree  granted  but  accredited  academic  institutions  are  linked  online 
and  supply  centralized  or  coordinated  services  to  students  with  no  articulation  among  consortium  members. 
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University  Information  Consortium-  no  degree  granted  and  no  centralized  or  coordinated  services  to 
students  but  accredited  academic  institutions  are  linked  electronically. 

Virtual  Program  - degree  granted  without  coming  to  a campus  from  a traditional  accredited  academic 
institution  where  the  majority  of  services  are  still  offered  in  a face-to-face  environment. 

Virtual  Commercial  Certification  Institution  - Certification  granted  but  no  academic  credit  awarded. 
Traditional  Accredited  Institution  with  Some  Electronic  Courses  - Credit  is  awarded  but  there  is  no 
coherence  among  electronically  offered  courses  that  could  lead  to  a degree  (Wolf  & Johnstone,  1999,  p.2) 

Applying  Wolf  and  Johnstone’s  Institutional  Taxonomy 

A Public  Virtual  University  Classification  Matrix  (Table  1)  was  developed  as  part  of  this  study  to  assist  in 
the  process  of  differentiating  between  the  different  types  of  public  virtual  universities  using  the  Wolf  and  Johnstone 
criteria.  The  matrix  was  created  using  the  eight  distinctive  characteristics  Wolfe  and  Johnstone  described  in  their 
classification  system: 

1.  Institutions  of  origin  linked  by  a Virtual  University. 

2.  Independent  institutions  or  non-for-profit  virtual  universities 

3.  Centralized  or  coordinated  student  services 

4.  Includes  mutual  course  articulation  and/or  transfer  agreements 

5.  No  mutual  course  articulation  and/or  transfer  agreement 

6.  Degree  granted  by  institution  of  origin 

7.  Degree  granted  by  virtual  university 

8.  Competency-based  credit  processes  in  place 

Information  for  each  of  the  nineteen  public  virtual  universities  was  examined  and  the  presence  or  absence 
of  these  characteristics  noted.  Institutions  were  then  classified  based  on  these  characteristics  into  one  of  the  seven 
categories  in  the  Wolfe  and  Johnstone  Institutional  Taxonomy.  A review  of  matrix  reveals  that  Western  Governors 
University  was  the  only  public  virtual  university  in  the  United  States  that  could  be  classified  under  the  Virtual 
University  / College  typology.  Two  public  virtual  universities  were  classified  under  the  Virtual  University 

typology.  These  two  virtual  universities  were,  the  SUNY  Learning  Network  and  the  University  of  Texas 
Telecampus.  Five  public  virtual  universities  were  classified  under  the  Academic  Services  Consortium  typology. 
These  five  virtual  universities  included  the  California 


Table  1:  Public  Virtual  University  Classification  and  Selection  Matrix 


Institution 

A 

B 

c 

D 

E 

F 

G 

H 

Classification 

Selected 

Arizona 

Regents 

X 

■ 

■ 

X 

■ 

X 

University  Information 
Consortium 

No 

California 

Virtual 

X 

X 

X 

X 

Academic  Services 
Consortium 

Yes 

Florida  Virtual 

X 

X 

X 

X 

Academic  Services 
Consortium 

Yes 

Georgia  Globe 

X 

■ 

■ 

■ 

X 

X 

University  Information 
Consortium 

No 

Illinois  Virtual 

X 

■ 

■ 

■ 

X 

X 

University  Information 
Consortium 

No 

Indiana 

Network 

X 

■ 

■ 

■ 

X 

X 

University  Information 
Consortium 

No 

Kentucky 

Virtual 

X 

X 

X 

X 

Academic  Services 
Consortium 

Yes 

Maryland 

Online 

X 

■ 

■ 

■ 

X 

X 

University  Information 
Consortium 

No 

Michigan 
Virtual  “ 

X 

X 

X 

X 

X 

Virtual  University 
Consortium 

No 

Minnesota 

Virtual 

X 

X 

X 

X 

Academic  Services 
Consortium 

Yes 

New  Jersey 
Virtual 

X 

■ 

■ 

■ 

X 

X 

University  Information 
Consortium 

No 

Ohio  Learning 
Network 

X 

■ 

■ 

■ 

X 

X 

University  Information 
Consortium 

No 
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Online  College 
Oklahoma 

X 

■ 

■ 

■ 

X 

X 

University  Information 
Consortium 

No 

South  Dakota 
Electronic 

X 

■ 

■ 

■ 

X 

X 

University  Information 
Consortium 

No 

Southern 

Regional 

X 

■ 

■ 

X 

■ 

X 

University  Information 
Consortium 

No 

SUNY  Learning 
Network 

X 

X 

X 

■ 

X 

Virtual  University 
Consortium 

Yes 

Tennessee 

Virtual 

X 

■ 

■ 

■ 

X 

X 

University  Information 
Consortium 

No 

UT: 

Telecampus 

X 

X 

X 

■ 

X 

Virtual  University 
Consortium 

Yes 

Western 

Governors 

■ 

■ 

X 

X 

■ 

■ 

X 

X 

Virtual 

University/College 

No 

Characteristics:  A=institutions  of  origin  linked  by  virtual  university;  B=independent  institution  or  non-for-profit 
virtual  university;  C=centralized  or  coordinated  student  services;  D=mutual  course  articulation  and/or  transfer 
agreements;  E=no  mutual  course  articulation  and/or  transfer  agreements;  F=degree  granted  by  institution  of  origin; 
G=degree  granted  by  virtual  university;  H=competency-based  credit  processes 
“ - review  of  the  data  indicate  that  this  institution  operates  as  a private  rather  than  public  entity. 

Virtual  University,  the  Florida  Virtual  Campus,  the  Kentucky  Virtual  University,  the  Michigan  Virtual 
University  and  the  Minnesota  Virtual  University.  The  University  Information  Coniort/Mm  typology  contained  the 
largest  number  of  public  virtual  universities.  This  typology  contained  ten  institutions.  The  institutions  that  were 
classified  in  this  category  included  the  Arizona  Regents  University,  the  Georgia  GLOBF,  the  Illinois  Virtual 
Campus,  the  Indiana  College  Network,  MarylandOnline,  the  New  Jersey  Virtual  University,  the  Ohio  Learning 
Network,  the  Online  College  of  Oklahoma,  the  South  Dakota  Electronic  University  Consortium,  and  the  Southern 
Regional  Electronic  Campus.  Each  of  the  remaining  three  virtual  university  typologies  from  the  Wolf  and  Johnstone 
Institutional  Taxonomy  did  not  contain  any  of  the  public  virtual  universities  in  the  United  States  (Wolf  and 
Johnstone,  1999). 

Following  the  review  of  the  public  virtual  institutions  using  the  Wolf  and  Johnstone  criteria,  the  institutions 
classified  as  Virtual  University  Cosortium  and  Academic  Services  Consortium  were  selected  to  examine  more 
closely.  The  public  virtual  universities  that  were  included  in  the  policy  analysis  component  of  this  study  were  all 
virtual  universities  that  1)  were  created  as  a collaborative  virtual  institution  that  linked  accredited  academic 
institutions  online,  2)  supplied  centralized  or  coordinated  services  to  students,  3)  provided  either  a direct  or  indirect 
articulation  among  consortium  members,  and  4)  provided  degree  granting  opportunities  at  an  institution  of  origin 
within  the  consortium  of  participating  institutions.  Seven  institutions  met  these  criteria.  Of  these,  one  institution  was 
found  to  be  operating  as  a private  entity  and  was  deleted  from  the  mix  (Michigan  Virtual  University).  This  choice 
resulted  in  the  selection  of  six  institutions  for  further  policy  analysis.  These  six  institutions  included:  the  California 
Virtual  University  (CVU),  the  Florida  Virtual  Campus  (FVC),  the  Kentucky  Commonwealth  Virtual  University 
(KYVU),  the  Minnesota  Virtual  University  (MNVU),  the  SUNY  Learning  Network  (SEN),  and  the  University  of 
Texas  Telecampus  (UT:Telecampus). 

To  examine  their  policies  more  closely,  it  was  important  to  first  select  a policy  analysis  framework  to  use 
as  a lens  through  which  to  look  at  these  institutions. 

Development  of  a Policy  Analysis  Framework 

A policy  analysis  framework  can  provide  guidance  in  looking  at  policy  structures,  identifying  essential 
policy  areas,  activities  and  processes  that  may  be  fundamental  to  the  operation  of  a public  institution  of  higher 
education.  Four  analysis  frameworks  found  in  the  literature  were  reviewed,  culminating  in  the  development  of  a 
framework  for  use  in  this  study.  The  components  of  the  four  frameworks  are  compared  in  table  2.  Berg’s  (1998) 
study.  Public  Policy  on  Distance  Learning  in  Higher  Education:  California  State  University  and  Western  Governors 
Association  Initiatives  utilized  a policy  analysis  framework  that  consisted  of  seven  constructs.  Berg’s  study  was 
conducted  in  1998  during  the  early  stages  of  the  virtual  university  development  process.  A number  of  the  constructs 
in  the  Berg  study  were  clearly  focused  on  the  feasibility  of  a virtual  university.  It  should  also  be  noted  that  the  Berg 
study  was  focused  specifically  on  the  policy  differences  between  Western  Governors  University  and  the  California 
Virtual  University. 

In  1998  Gellman-Danley  and  Fetzner  published  an  article  in  the  Online  Journal  of  Distance  Learning 
Administration  entitled  Aikmg  the  Really  Tough  Questions:  Policy  Issues  for  Distance  Learning.  In  this  article 
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Gellman-Danley  and  Fetzner  proposed  a seven  element  model  to  assist  distance  education  decision-makers  with 
policy  issues.  In  an  article  published  later  in  1998,  entitled  Barriers  to  Online  Teaching  in  Post-Secondary 
Institutions:  Can  Policy  Changes  Fix  Itl,  Berge  added  two  areas  to  the  policy  framework  that  was  developed  by 
Gellman-Danley  and  Fetzner. 

In  the  spring  of  2000  a team  of  faculty  members  and  graduate  students  from  the  University  of  Nebraska 
published  a paper  in  the  Online  Journal  of  Distance  Learning  Administration  entitled  Policy  Frameworks  for 
Distance  Education:  Implications  for  Decision  Makers.  This  policy  framework  was  designed  to  assist  decision- 
makers with  the  growing  number  of  policy  issues  in  the  distance  education  arena  (King,  Nugent,  Russell,  Eich, 
Lacy,  2000).  This  group  adapted  policy  frameworks  of  Gellman-Danley  and  Fetzner  (1998)  and  Berge  (1998) 
hoping  to  create  a policy  analysis  framework  for  distance  education  that  could  be  adapted  and  generalized  for  a 
variety  of  post-secondary  environments  including  four  year  colleges,  community  colleges,  private  universities  and 
public  universities. 

The  policy  categories  that  were  selected  to  be  included  in  this  study  were  present  in  at  least  three  of  the 
four  distance  learning  frameworks  that  had  been  previously  developed  (see  Table  2).  The  exceptions  to  this  rule  are 
the  ‘Private  Industry’  policy  category  and  the  ‘Access  to  Education’  policy  category.  The  ‘Private  Industry’  policy 
category  was  included  in  the  Public  Virtual  University  Policy  Analysis  Eramework  based  on  the  Rosevear’s  (1999) 
examination  of  the  virtual  university  and  the  need  to  develop  partnerships  with  private  industry.  Rosevear  (1999) 
argued  that  virtual  universities  need  to  develop  partnerships  with  the  private  sector  in  order  to  be  successful  in 
meeting  the  needs  of  the  community  and  the  industries  that  the  virtual  universities  serve. 


Table  2:  Comparison  of  policy  categories  from  previous  studies. 


Policv  Categories 


Gellman-Danley 
& Eetzner 


Private  industry 


Competency-based 


Governance 


Labor/Management/ Administration 


Accreditation 


Training 


Eunding/Fiscal 


Consumerism 


Academic 


Geographic 


Legal 


Student  Support  Services 


Technical 


Cultural 


Note:  Shaded  categories  were  included  in  framework  developed  for  this  study. 


King  et.al 


The  ‘Access  to  Education’  policy  category  was  added  to  examine  the  issue  of  access  to  education,  an 
element  appearing  in  public  venues  as  a primary  reason  for  developing  and  marketing  online  education  such  as  that 
offered  through  virtual  universities.  The  resulting  policy  analysis  framework  used  in  this  study  is  described  below. 


Academic  Admission  Standards,  Admissions  Process,  Course/Program/Degree  Availability,  Course 

Integrity,  Curriculum/Course  Approval,  Accreditation,  Course  Logistics,  Eaculty 
Development  and  Resources 

Access  to  Education  Access  to  Technology,  Access  to  Internet  Resources,  Scholarships,  Policies  that  Support 
Learner-Centered  Instruction  and  the  Needs  of  Diverse  Students,  Instruction  Models  that 
Allow  Students  to  Work  in  Small  Groups,  and  Curriculums  Organized  Around  Central 
Concepts 

Funding  / Fiscal  Fees,  State  Fiscal  Regulations,  Tuition  Disbursement,  Tuition  Rate  Adjustments, 

Legislative  Funding,  Financing,  Grants 

Governance/  Staffing,  Institutional  Communication,  Technology,  Institutional 

Administration  Technical  Support,  Faculty  and  Personnel  Issues,  Board  Oversight,  Union  Contracts, 

Mission,  Planning 
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Private  Industry  Partnerships,  Support,  Research 

Student  Services  Availability,  Library  Services,  Registration,  Financial  Aid,  Student  Accounts, 

Counseling,  Academic  Advising,  Assessment  Testing,  Test  Proctoring,  Student  Records, 
Student  Technical  Support 

Data  Collection  and  Analysis 

Six  types  of  policy  documents  were  selected  to  be  included  in  this  examination. 

These  were  common  documents  that  appeared  to  be  readily  available  from  the  majority  of  public  virtual  universities. 
The  policy  documents  collected  from  each  of  the  selected  public  virtual  universities  included:  enabling  legislation, 
funding  legislation,  institutional  / operational  documents,  statewide  or  regional  governing  board  documents,  internal 
and  external  press  releases,  and  journal  / magazine  articles.  The  list  below  defines  the  selected  documents. 

Euabliug  Legislatiou  Documents  related  to  the  creation  of  the  virtual  university.  These  policy 

documents  may  be  from  either  state  legislation  or  regional  consortia. 
Funding  Legislation  Documents  related  to  the  funding  of  a virtual  university  can  originate 

from  a state  legislature,  a regional  consortium  of  states,  public 
institutions  of  higher  education,  a state  board  of  higher  education, 
state/federal  granting  agencies,  private  funding  agency,  or  private 
partnerships. 

Institutional/Operational  Documents  related  to  the  operational  policies  and 

Documents  procedures  of  the  virtual  university.  Examples  of  these  documents 

would  include  the  mission  statement,  goals,  strategic  planning 
documents,  student  handbook  or  any  other  document  related  to  the 
operation  of  the  virtual  university. 

Statewide  or  Regional  Governing  Documents  originating  from  a statewide  or  regional 

Board  Documents  governing  board.  These  documents  may  focus  on  the  development  or 

operation  of  the  virtual  university. 

Internal  and  External  Documents  released  in  the  general  press  regarding  a 

Press  Releases  specific  virtual  university  or  an  examination  of  a group  of  virtual 

universities.  These  documents  may  originate  from  an  independent 
press  agency  or  from  the  public  relations  department  of  a virtual 
university. 

Journal  and  Magazine  Documents  originating  from  either  a journal  or  magazine  article.  These 

articles  may  be  either  technical  in  nature  or  descriptive.  These  articles 
many  also  originate  from  a professional  news  source. 

Several  processes  were  used  to  collect  these  policy  documents,  including  a detailed  examination  of  the 
web  sites  related  to  the  public  virtual  universities,  including  the  web  sites  for  the  state  or  regional  governing  boards, 
and  a review  of  all  online  distance-learning  journals.  Primary  library  databases,  including  Lexus/Nexus,  were 
searched  for  any  references  to  the  virtual  universities  included  in  the  study.  A policy  document  request  letter  was 
sent  to  the  public  relations  contact  for  each  public  virtual  university  requesting  all  previously  outlined  policy 
documents.  This  was  followed  up  when  necessary  with  sending  a policy  document  request  letter  to  the  chief 
executive  officer  or  president  at  each  public  virtual  university.  Qualitative  document  analysis  procedures  were  used 
to  further  examine  the  collected  documents.  Data  from  the  six  public  virtual  universities  examined  in  this  study  were 
grouped  according  to  the  policy  categories. 

Findings 

The  policy  documents  were  reviewed  and  highlighted  according  to  the  themes  of  the  policy  categories  from 
McCoy’s  Public  Virtual  University  Policy  Analysis  Framework.  Tables  of  policy  categories  were  then  created  for 
each  institution.  These  tables  were  then  used  to  discover  the  existence  of  patterns  in  policy  characteristics  within 
each  of  the  public  virtual  universities  and  across  institutions.  The  researcher  determined  that  a pattern  existed  if 
three  or  more  institutions  contained  the  same  emphasis  on  a specific  policy  characteristic. 
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Patterns  in  Policy  Characteristics  Related  to  the  Academic  Policy  Category 

A single  finding  emerged  from  an  examination  of  the  text  data  related  to  the  academic  policy  category. 

This  finding  is  related  to  the  availability  of  faculty  development  and  training  resources  and  emerged  from  the  policy 
documents  from  three  of  the  six  public  virtual  universities  included  in  this  study. 

Three  of  the  institutions  included  in  the  study  developed  and  provided  faculty  development,  faculty  training 
opportunities,  and  faculty  resources:  the  UT  TeleCampus,  SLN,  and  the  KYVU.  These  three  public  virtual 
universities  also  reported  high  levels  of  enrollments.  Therefore,  this  finding  may  suggest  a relationship  between  the 
availability  of  faculty  resources  and  increased  annual  enrollments.  Patterns  in  policy  characteristics  from  the 
academic  policy  category  are  summarized  in  Table  3. 

Table  3:  Summary  of  Characteristics  Related  to  Academic  Policies  for  Six  Selected  Public  Virtual  Universities  in 
the  United  States 


Institutions 

Policy  Characteristics 

CVU 

No  mention  of  academic  policies  related  to  faculty  development. 

FVC 

No  mention  of  academic  policies  related  to  faculty  development. 

KYVU 

Workgroups  in  place  and  faculty/staff  advise  director.  Faculty  development 
program  available  and  faculty  discussion  forums  provided. 

MNVU 

No  mention  of  academic  policies  related  to  faculty  development. 

SLN 

Training  and  technical  support  provided  for  faculty. 

UT 

Telecampus 

Assistance  for  faculty  with  course  development  and  delivery  and  intellectual 
property  and  copyright  issues.  Provides  an  online  library. 

Academic:  admission  standards,  admission  process,  course/program/degree  availability,  course  integrity, 
curriculum/course  approval,  accreditation,  course  logistics,  faculty  development,  and  resources. 

Patterns  in  Policy  Characteristics  Related  to  the  Access  to  Education  Policy  Category 

The  text  data  collected  from  all  six  virtual  universities  revealed  that  five  of  the  six  virtual  universities  had 
data  that  supported  a central  theme  of  providing  increased  access  to  education.  The  data  for  CVU  are  lacking  a 
foundation  in  the  concept  of  providing  increased  access  to  education.  This  finding  suggested  a potential  relationship 
between  virtual  universities  that  contained  a central  theme  of  providing  increased  access  to  education  and  surviving 
virtual  universities. 

In  CVU  policy  documents,  the  three  issues  that  drove  the  development  of  the  CVU  included  economic 
development,  exporting  to  global  market,  and  dealing  with  influx  of  new  students  in  California  rather  than 
expanding  access  to  new  groups  of  students.  Table  4 summarizes  the  access  to  education  policies  found  in  the 
policy  documents. 

Table  4:  Summary  of  Characteristics  Related  to  Access  to  Education  Policies  for  Six  Selected  Public  Virtual 
Universities  in  the  United  States 


Institutions 

Policy  Characteristics 

CVU 

Absorb  additional  students  based  on  demographic  bulge,  economic  development 

FVC 

Affordable  access  to  quality  learning  opportunities  to  citizens,  seamless  distance 
learning  experience,  address  welfare  to  work  programs. 

KYVU 

Make  postsecondary  education  more  accessible,  efficient,  and  responsive  to  citizens; 
help  more  citizens  realize  their  educational  dreams;  increase  educational  attainment 

MNVU 

Make  higher  education  more  accessible  to  citizens  of  the  state 

SLN 

Provide  accessible,  affordable,  high-quality  education  for  citizens;  help  citizens  fulfill 
educational  goals 

UT 

Telecampus 

Provide  equitable  distribution  of  higher  education  opportunities  to  all  citizens;  increase 
access  to  education;  and  build  intellectual  capital,  workforce  development 

Access:  access  to  technology,  access  to  Internet  resources,  scholarships,  policies  that  support  learner- 
centered  instruction  and  the  needs  of  diverse  students,  instruction  models  that  allow  students  to  work  in  small 
groups,  application  of  team  teaching  concepts,  curricula  organized  around  central  concepts, assessment  services 
provided  based  on  social  stratifications  and  structural  basis  of  dominance. 
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Patterns  in  Policy  Characteristics  Related  to  the  Funding/Fiscal  Policy  Category 

Three  patterns  emerged  from  the  policy  analysis  process  related  to  the  funding/  fiscal  policy  category.  The 
first  was  related  to  initiating  agency  providing  the  funding  sources  and  the  variation  of  funding  sources  among  the 
individual  virtual  universities.  This  finding  indicated  that  funding  sources  and  the  availability  of  alternative  funding 
for  the  public  virtual  universities  might  differ  based  on  the  type  of  statewide  postsecondary  structure.  The  second 
finding  indicated  a potential  relationship  between  the  level  of  funding  at  the  public  virtual  universities  and  increased 
enrollments.  The  third  finding  was  related  to  the  funding  of  CVU.  This  finding  indicated  that  while  the  funding 
level  for  CVU  was  comparable  to  the  funding  that  the  other  public  virtual  universities  received,  there  were 
differences  in  funding  sources  that  may  have  implications  for  policy. 

First,  initial  funding  sources  for  the  public  virtual  universities  appeared  to  vary  based  on  the  type  of 
statewide  postsecondary  structure.  Four  of  the  public  virtual  universities  included  in  this  study  were  based  on  an 
open  postsecondary  system.  These  institutions  included  CVU,  MNVU,  the  FVC,  and  KYVU.  The  public  virtual 
universities  in  these  states  were  initiated  by  a legislative  act.  The  annual  funding  levels  for  these  public  virtual 
universities  varied  from  $350,000  to  $3.5  million.  These  institutions  were  all  classified  as  Academic  Services 
Consortium  based  on  the  Wolf  and  Johnstone  (1999)  Institutional  Taxonomy. 

Two  of  the  public  virtual  universities  included  in  this  study  were  based  on  integrated  statewide  systems  for 
postsecondary  education.  These  institutions  include  the  UT  TeleCampus  and  the  SUN.  The  public  virtual 
universities  in  these  states  were  initiated  by  the  boards  of  regents  that  represented  all  of  the  public  postsecondary 
institutions  of  higher  education  in  the  specific  states.  Funding  in  these  cases  relied  heavily  on  institutional 
contributions.  The  annual  funding  levels  for  these  public  virtual  universities  varied  from  $1.2  million  to  $3.4 
million.  These  two  institutions  were  classified  as  Virtual  University  Consortia  based  on  the  Wolf  and  Johnstone 
(1999)  Institutional  Taxonomy.  While  the  initiating  agencies  differed  by  postsecondary  structure,  the  funding  level 
for  each  of  the  public  virtual  universities  included  in  the  study  appears  to  depend  on  neither  the  type  of 
postsecondary  system  that  existed  in  each  of  the  states  nor  the  type  of  virtual  institution  (Academic  Service 
Consortium  or  Virtual  University  Consortium 

Four  of  the  public  virtual  universities  included  in  this  study  received  alternative  funding  for  the  sale  of 
services,  fromcorporate  partnerships,  or  from  education  foundations.  The  funding  from  these  sources  was  added  to 
the  overall  operational  budget  of  each  of  these  institutions.  The  levels  of  alternative  funding  varied  from  $4.1 
million  at  the  SLN  to  in-kind  contributions  at  the  MNVU.  Alternative  funding  for  the  public  virtual  universities  did 
not  appear  to  depend  on  the  type  of  postsecondary  educational  structure  in  the  sponsor  state  and  neither  did  the 
survival  of  the  institution  seem  to  depend  on  the  level  of  alternative  funding. 

A second  finding  indicated  a possible  relationship  between  the  annual  level  of  funding  for  the  public  virtual 
universities  and  increased  enrollments.  The  three  virtual  universities  included  in  this  study  that  reported  the  highest 
levels  of  institutional  and/or  alternative  funding  are  also  the  three  institutions  that  reported  the  highest  levels  of 
annual  enrollments. 

A third  finding  relates  to  CVU.  The  California  state  legislature  initially  provided  funding  comparable  to  the 
level  of  funding  that  the  other  public  virtual  universities  received.  Although  the  state  funding  for  CVU  was 
discontinued  during  the  second  year  of  operation,  CVU  was  able  to  generate  alternative  sources  of  revenue  that  were 
as  large  as  the  state  funding  levels  for  the  FVC  or  MNVU. 

While  the  funding  level  does  not  seem  to  be  the  key  to  understanding  the  fate  of  CVU,  examining  the 
source  of  the  funding  might  provide  better  insights.  When  CVU’s  state  funding  was  withdrawn  during  the  second 
year  of  operation,  the  virtual  institution  was  reliant  upon  corporate  sources  of  funding  and  grants  from  foundations 
and  was  told  to  operate  as  a nonprofit  institution  without  state  funds.  The  SLN  was  the  only  other  virtual  institution 
included  in  the  policy  analysis  component  of  this  s tudy  that  had  significant  reliance  upon  foundation  sources  of 
support.  However,  unlike  CVU,  the  SLN  was  able  to  maintain  the  core  support  from  the  state  in  addition  to  the 
funding  it  received  from  the  Sloan  Foundation.  CVU  was  the  only  virtual  institution  whose  operational  budget  was 
primarily  funded  through  corporate  and  foundation  contributions  after  the  initial  support  from  the  state. 

What  is  not  documented  in  these  policy  documents  from  CVU  is  the  requirements  or  constraints  connected 
to  the  alternative  funding  that  CVU  received  from  the  corporations.  Additional  research  related  to  the  dissolution  of 
CVU  might  investigate  the  contingencies  placed  on  CVU  by  the  corporations  and  foundations  that  provided  the 
alternative  funding  once  CVU  became  a not-for-profit  entity.  The  issue  of  generating  revenue  from  advertising  and 
book  sales  adds  to  the  concern  about  the  status  of  the  corporations  that  had  provided  alternative  funding  for  CVU. 

Among  the  six  institutions  studied,  four  were  initially  supported  from  state  revenues.  Two,  the  SLN  and 
the  UT  TeleCampus,  were  initially  supported  institutionally  or  from  the  Board  of  Regents  with  additional  sources  of 
funding  provided  from  foundations  or  sales  of  services.  Amounts  of  funding  for  these  institutions  varied  widely. 
Among  the  six,  CVU  was  the  only  public  virtual  institution  whose  funding  from  the  state  was  eliminated  and  the 
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institution  became  funded  through  corporate  and  foundation  donations.  These  data  indicate  that  the  fiscal  factor  in 
the  dissolution  of  the  CVU  was  not  due  to  the  lack  of  financial  support  but  perhaps  due  to  the  lack  of  state 
commitment.  The  difference  between  hard  money  (state  revenue  supported)  and  soft  money  (corporate  and 
foundation  supported)  may  have  provided  unique  challenges  to  CVU.  These  data  may  indicate  a need  to  examine 
the  commitment  of  public  legislative  or  governing  bodies  as  a policy  area  connected  to  the  success  of  virtual 
institutions.  Table  5 summarizes  findings  related  to  the  funding/fiscal  policy  area. 

Table  5:  Summary  of  Characteristics  Related  to  Funding/Fiscal  Policies  for  Six  Selected  Public  Virtual  Universities 
in  the  United  States 


Institutions 

Policy  Characteristics 

CVU 

Initially  state  funded  ($6.1  million),  shifted  to  bulk  of  funding  from  corporate 
sponsors  ($2  million)  and  some  from  Alfred  P.  Sloan  Eoundation  ($625,000). 
Told  to  operate  as  a nonprofit  organization  without  state  funding. 

EVC 

Eunded  through  state  budget  at  $375,000. 

KYVU 

Eunded  through  state  budget  at  $8  million. 

MNVU 

Eunded  primarily  through  state  budget  ($1  million)  with  some  in-kind 
contributions  from  partners. 

SEN 

Half  funded  through  the  Sloan  Eoundation  ($1.2  million  annually)  with  the  other 
half  consisting  of  in-kind  contributions  from  the  SUNY  system. 

UT 

TeleCampus 

Eunded  primarily  by  the  Board  of  Regents  ($3.4  million)  with  some  contributions 
from  sales  of  services. 

Funding/Fiscal:  fees,  state  fiscal  regulations,  tuition  disbursement,  tuition  rates  and  adjustments,  legislative 
funding,  financing  grants. 

Patterns  in  Policy  Characteristics  Related  to  the  Governance/Administration  Policy  Category 

Three  significant  findings  emerged  from  the  examination  of  documents  related  to  the 
governance/administration  policy  category.  First,  five  of  the  six  public  virtual  universities  utilized  some  form  of 
strategic  planning  process  during  the  development  phase  of  the  virtual  university,  and  completed  similar  steps 
during  the  development  phase. 

Second,  each  of  these  five  public  virtual  universities  utilized  some  sort  of  planning  committee  to  develop  a 
strategic  or  master  plan.  The  CVU  Design  Team,  established  by  executive  order,  developed  a blueprint  for  the 
CVU.  Florida  Distance  Learning  Institute  established  the  Virtual  Design  Team  to  develop  the  recommended 
structure  and  design  of  the  FVC.  The  Council  of  Postsecondary  Education  was  responsible  for  the  development  of 
KYVU.  The  Kentucky  Council  of  Postsecondary  Education  developed  the  organizational  structure,  policies,  and 
procedures  related  to  the  development  of  KYVU.  Eour  design  committees  were  responsible  for  the  development  of 
MNVU,  a coordinating  board,  student  services  design  team,  a quality-in-learning  team,  and  a technology  team.  The 
UT  system  developed  the  Distance  Education/Virtual  University  Master  Plan  Organization  to  prepare  the  UT 
TeleCampus  Master  Plan. 

Each  public  virtual  university  established  a committee  or  council  to  develop  the  strategic  plans  or  master 
plans  that  led  to  the  development  of  these  institutions.  Although  the  process  utilized  by  each  institution  was  unique, 
they  were  all  similar  in  that  they  utilized  key  academic  and  political  stakeholders  to  guide  the  development  of  the 
virtual  university.  The  process  that  CVU  utilized  for  planning  and  development  was  similar  to  comparable  public 
virtual  universities. 

The  third  finding  pointed  to  the  importance  of  a mission  statement  or  an  institutional  goal  statement.  Each 
of  the  institutions  included  in  the  study  created  and  operationalized  some  form  of  mission  statement  or  institutional 
goal  during  the  development  phase  of  the  virtual  university. 

The  contrast  between  the  mission  and  goal  statements  for  the  six  public  virtual  universities  is  also  notable. 
Three  of  the  institutions,  the  EVC,  KYVU,  and  MVU,  had  similar  mission  statements  in  that  they  each  promoted 
availability  of  resources  and  increased  access  to  postsecondary  educational  opportunities.  The  mission  and  goal 
statements  from  CVU,  the  SEN,  and  the  UT  TeleCampus  were  based  on  the  concepts  of  technology,  innovation,  and 
the  economies  of  scale  inherent  in  distance-learning  initiatives.  Although  there  is  considerable  contrast  in  the 
content  of  the  mission  statements  or  the  institutional  goal  statements,  the  significance  of  this  finding  is  the 
completion  of  the  mission  or  goal-setting  process.  Table  6 summarizes  findings  related  to  governance  and 
administration. 
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Table  6:  Summary  of  Characteristics  Related  to  Governance/Administration  Policies  for  Six  Selected  Public  Virtual 
Universities  in  the  United  States. 


Institutions 

Policy  Characteristics 

CVU 

Blueprint,  design  team  from  higher  education  colleges  and  universities.  Goal  is  to 
tie  together  online  and  distance-education  offerings  of  the  colleges  and  universities, 
help  students  find  courses,  and  assist  schools  in  marketing  online  education. 

FVC 

Design  team.  Goal  is  to  provide  comprehensive  information  resources,  to  expand 
access  and  availability,  and  to  respond  to  needs  of  higher  education. 

KYVU 

Council  on  postsecondary  education  developed  policies  and  design  structure,  higher 
education  institutions  consulted.  Goal  is  student-centered  coordination  of  delivery 
of  distance  education  to  meet  citizen  and  employer  needs. 

MNVU 

Four  design  committees  (coordinating  board,  student  services  and  design,  quality  in 
learning,  and  technology).  Committees  included  administrators,  faculty,  student 
services  personnel,  students,  and  private-sector  representatives.  Goal  is  to  connect 
learners,  industry,  and  education  as  partners  to  build  economic  stability  and  to 
facilitate  access,  quality,  partnerships,  and  offerings. 

SEN 

Board  of  Regents.  Oversight  goal  is  to  focus  on  accountability  and  learning 
initiatives  of  the  academic  institutions,  to  provide  access  to  students.  Mission  of 
providing  increased  accessibility  for  students. 

UT:  TeleCampus 

Master  plan,  team  of  faculty,  staff,  students,  and  a task  force. 

Governance/Administration:  staffing,  institutional  communication,  technology,  institutional  technical  support, 
faculty  personnel  issues,  board  oversight,  union  contracts,  mission,  planning. 


Patterns  in  Policy  Characteristics  Related  to  the  Private  Industry  Policy  Category 

An  examination  of  the  data  related  to  the  private  industry  policy  category  did  not  reveal  any  patterns  related 
to  the  private  industry  policy  category.  Although  a pattern  did  not  emerge  there  was  one  noteworthy  finding  related 
to  the  private  industry  policy  category.  Five  of  the  six  public  virtual  universities  did  not  maintain  significant 
partnerships  with  private  industry  corporations.  The  CVU  was  the  only  public  virtual  university  that  maintained 
partnerships  with  private  corporations;  the  CVU  also  received  funding  related  to  the  partnerships  from  private 
corporations. 

The  role  of  private  industry  as  related  to  the  virtual  university  was  an  ongoing  theme  in  the  literature 
reviewed  for  this  study.  It  may  be  an  important  finding  in  that  five  of  the  six  public  virtual  universities  examined  in 
this  study  did  not  rely  on  developing  or  maintaining  relationships  with  private  industry  beyond  the  relationships  that 
they  had  developed  with  the  vendors  who  provided  the  technological  means  for  the  delivery  of  courses  and  services. 
It  may  also  be  a useful  finding  that  only  the  non-surviving  public  virtual  university  developed  dynamic  partnerships 
with  private  industry  that  had  been  designed  to  provide  revenue  models  for  the  corporate  sponsors  and  partners. 

Patterns  in  Policy  Characteristics  Related  to  the  Student  Services  Policy  Category 

Two  significant  findings  emerged  from  the  examination  of  the  data  related  to  the  student  service  policy 
category.  The  first  finding  revealed  that  the  number  of  references  to  student  service  that  emerged  from  the  CVU 
policy  documents  was  significantly  smaller  than  those  from  the  documents  of  the  other  five  virtual  universities 
included  in  this  study.  The  second  finding  revealed  that  four  out  of  the  six  public  virtual  universities  provided 
quality,  comprehensive  student  services  in  a “one-stop  shop”  environment.  The  data  related  to  this  finding  suggest 
that  this  “one-stop  shop”  format  for  providing  student  services  may  be  critical  to  the  success  of  distance-learning 
initiatives. 

When  compared  to  the  other  five  virtual  universities  included  in  the  policy  analysis,  it  was  evident  that 
CVU  had  a less  clear  focus  on  the  development  of  comprehensive  student  services.  Although  the  CVU  policy 
documents  demonstrated  intent  to  develop  comprehensive  student  services,  it  is  also  clear  that  the  development  of 
student  services  was  not  a priority  at  CVU.  In  fact,  CVU  student  services  were  not  yet  operational  at  the  time  of  the 
dissolution.  Comparably,  the  UT  TeleCampus  developed  the  student  service  functions  of  the  TeleCampus  before  it 
began  to  provide  access  to  distance-learning  courses. 

An  analysis  of  data  related  to  student  services  from  the  other  five  virtual  universities  demonstrated  a 
commitment  to  the  development  of  comprehensive  student  services  and  a focus  on  quality  of  services.  Each  of  the 
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remaining  five  virtual  universities  included  in  this  study  developed  a method  or  means  to  deliver  comprehensive 
student  services  to  their  distance-learning  students. 

The  second  finding  pointed  to  a theme  of  providing  access  through  a one-stop,  single-point-of-access 
model.  FVC  was  designed  to  “ensure  one-stop  shopping  for  distance-learning  students.’’  KYVU  has  a policy  of 
“providing  one-stop  services  on  the  Worldwide  Web’’  to  students.  MNVU  was  designed  to  “offer  streamlined 
access  to  financial  aid  information  and  services;  one-stop  registration  and  tuition  services.’’  The  UT  TeleCampus 
“worked  to  give  the  distance  education  student  every  available  chance  to  succeed,  along  with  the  equivalent  of  one- 
stop  shopping.’’  Four  of  the  six  public  virtual  universities  concluded  that  providing  quality,  comprehensive  student 
services  in  a “one-stop  shop’’  environment  was  critical  to  the  success  of  their  distance-learning  initiative.  Table  7 
summarizes  findings  in  the  student  services  area. 

Table  7:  Summary  of  Characteristics  Related  to  Student  Services  Policies  for  Six  Selected  Public  Virtual 
Universities  in  the  United  States. 


Institutions 

Policy  Characteristics 

CVU 

Planned  for  the  development  of  integrated  student  services. 

FVC 

Quality,  accessible,  “one-stop  shop”  using  existing  resources. 

KYVU 

Single  application,  single  application  fee,  single  registration  form,  and  “one-stop 
shop.”  Web-based  support  services,  access  to  library,  and  academic  support. 

MNVU 

One-stop  registration  and  tuition  services,  streamlined  access  to  services  including 
financial  aid,  library  and  career  services. 

SEN 

Supports  the  campuses  working  together,  not  “one-stop  shop.”  Each  participating 
campus  provides  student  services  via  the  SEN. 

UT 

TeleCampus 

Individual  institutions  maintain  standards.  “One-stop  shop”  services  include 
admission,  registration,  financial  aid,  bookstore,  and  library. 

Student  services:  availability,  library  services,  registration,  financial  aid,  student  account, 

counseling,  academic  advising,  assessment  testing,  test  proctoring,  student  records,  student  technical  assistance. 


Conclusion 

This  study  generated  eight  noteworthy  conclusions.  These  conclusions  are  important  in  that  they  provide  a 
better  understanding  of  the  elements  of  the  policies  of  the  public  virtual  universities  when  examined  as  a whole. 
These  key  conclusions  are  both  independent  findings  generated  from  the  study  and  general  conclusions  that  can  be 
made  based  on  the  collective  examination  of  the  findings  from  this  study. 

Conclusion  1:  Application  of  the  Wolf  and  Johnstone  Taxonomy 

The  application  of  the  Wolf  and  Johnstone  Institutional  Taxonomy  provided  a useful  means  for  the 
classification  of  the  19  public  virtual  universities.  The  application  of  the  Wolf  and  Johnstone  Institutional 
Taxonomy  resulted  in  the  classification  of  one  institution  as  a Virtual  University/College,  three  as  Virtual  University 
Consortia,  four  institutions  as  Academic  Services  Consortia,  and  1 1 institutions  as  University  Information  Consortia. 
Conclusion  2:  Faculty  Development  and  Enrollment 

The  second  conclusion  is  that  there  may  be  a relationship  between  faculty  development  resources  and  the 
success  of  a public  virtual  university.  Three  of  the  public  virtual  universities  included  in  this  study  developed  and 
provided  opportunities  for  faculty  development,  faculty  training  in  distance-learning  methods,  and  access  to 
instructional  technology  resources. 

Conclusion  3:  Funding  Levels  of  Surviving  and  Non-surviving  Public  Virtual  Universities 

The  third  conclusion  is  that  there  appeared  to  be  no  significant  difference  in  the  funding  levels  of  surviving 
virtual  universities  and  a non-surviving  virtual  university.  However,  CVU  was  the  only  public  virtual  institution 
whose  funding  from  the  state  was  eliminated  and  which  then  became  funded  through  corporate  and  foundation 
donations.  The  data  indicated  that  the  fiscal  factor  in  the  dissolution  of  CVU  was  not  due  to  the  lack  of  financial 
support  but  perhaps  due  to  the  lack  of  state  commitment.  CVU  was  no  longer  supported  via  hard  money  (state 
revenue  supported)  and  became  dependent  on  soft  money  (corporate  and  foundation  supported)  to  continue 
operation.  Although  the  data  demonstrated  that  the  amount  of  soft  money  was  equal  to  or  greater  than  the  annual 
funding  of  comparable  institutions,  the  requirements  or  constraints  of  soft  money  funding  may  have  provided  unique 
complications  for  CVU  in  addition  to  the  lack  of  stability. 

This  conclusion  is  important  in  that  it  supports  the  explanation  of  the  failure  of  CVU  provided  by  the 
former  CEO  of  the  virtual  institution,  Stanley  A.  Chodorow.  In  an  issue  of  the  Chronicle  of  Higher  Education, 


293 


Chodorow  (cited  in  Young,  2000)  commented  on  the  financial  plan  of  CVU,  saying,  “The  original  plan  for  the  CVU 
did  not  include  an  adequate  financial  structure.’’  He  went  on  to  say  that  “there  was  a plan  to  raise  funds  from 
corporate  sponsors,  and  the  governor  had  helped  to  do  that,  but  he  had  only  gotten  five  or  so  out  of  10  that  were 
required  for  the  original  financial  plan.  And  once  the  governor  was  out  of  office,  these  corporations  were  not  very 
interested  in  continuing  their  support’’  (Young,  2000,  p.  1). 

Conclusbn  4:  Importance  of  Student  Services 

The  fourth  conclusion  is  that  there  is  a possible  relationship  between  the  development  of  comprehensive 
student  services  and  the  viability  of  public  virtual  universities.  This  conclusion  supports  the  argument  that 
comprehensive  student  services  are  necessary  to  support  the  efforts  of  distance-learning  students.  In  the  case  of 
CVU,  the  strategic  plan  provided  for  the  development  of  online  student  services  but  these  services  had  not  yet  come 
to  fruition  at  the  time  of  the  dissolution  of  CVU.  In  comparison,  the  UT  TeleCampus  developed  the  online  student 
services  portion  of  the  virtual  university  before  it  began  to  provide  access  to  distance-learning  courses. 

In  addition  to  the  development  of  comprehensive  student  services,  this  finding  also  indicated  that  the 
manner  in  which  the  student  services  were  provided  can  be  a significant  factor.  Four  of  the  six  public  virtual 
universities  provided  quality,  comprehensive  student  services  in  a “one-stop  shop’’  environment.  Each  of  these  four 
institutions  argued  that  this  “one-stop  shop”  format  was  critical  to  the  success  of  their  distance-learning  initiative. 
Conclusion  5:  Funding  Level  and  Enrollment 

The  fifth  conclusion  is  that  there  may  be  a relationship  between  the  level  of  funding  of  a public  virtual 
university  and  the  annual  student  enrollment  of  a public  virtual  university.  The  three  public  virtual  universities  that 
reported  the  highest  levels  of  annual  funding  from  both  primary  and  alternative  sources  also  reported  the  highest 
levels  of  annual  enrollments.  This  conclusion  points  to  a potential  relationship  between  the  level  of  annual  funding 
for  a public  virtual  university  and  the  level  of  annual  student  enrollment  at  a public  virtual  university. 

Conclusion  6:  Focus  on  Access  to  Education 

The  sixth  conclusion  is  that  the  central  theme  of  providing  access  to  education  may  be  critical  to  the 
development,  growth  and  success  of  a public  virtual  university.  All  surviving  public  virtual  institutions  had  a focus 
on  expanding  access  to  previously  unserved  or  underserved  populations,  as  opposed  to  CVU,  which  was  more 
focused  on  economic  development  and  managing  population  growth. 

Conclusion  7:  Relationship  with  Private  Industry 

The  seventh  conclusion  is  that  relationships  with  private  industry  do  not  seem  linked  to  success.  Five  of  the 
six  public  virtual  universities  examined  in  this  study  did  not  rely  on  developing  or  maintaining  a relationship  with 
private  industry  beyond  the  relationships  that  they  have  developed  with  the  vendors  who  provide  the  technological 
means  for  the  delivery  of  courses  and  services.  The  only  non-surviving  public  virtual  university  was  the  only  public 
virtual  university  to  develop  dynamic  partnerships  with  private  industry.  These  partnerships  had  been  designed  to 
provide  revenue  models  for  the  corporate  sponsors  and  partners.  This  finding  supports  the  argument  that  the  lack  of 
hard  money  support  from  the  state  and  reliance  on  corporate  sponsorship  may  be  significant  in  the  dissolution  of 
CVU. 

Conclusion  8:  Apparent  Differences  in  Surviving  and  Nonsurviving  Public  Virtual  Universities 

The  eighth  conclusion  that  can  be  extrapolated  from  this  study  is  that  the  failure  of  CVU  does  not  appear  to 
have  been  caused  by  the  lack  of  primary  or  alternative  funding  sources  or  amount  of  resources  provided.  Neither 
does  the  failure  of  CVU  seem  to  have  been  caused  by  the  lack  of  a comprehensive  strategic  planning  process.  Based 
on  this  research  study,  it  may  be  possible  that  the  failure  of  CVU  resulted  from  the  model  that  was  created  by  the 
stakeholders.  When  compared  to  the  other  public  virtual  universities  included  in  this  study,  CVU  was  found  to 
differ  on  several  fundamental  characteristics  that  the  more  successful  public  virtual  universities  contained. 

CVU  did  not  provide  comprehensive  student  services.  CVU  did  not  provide  comprehensive  faculty 
training  and  resources.  The  motivation  for  the  development  of  CVU  was  not  based  on  a central  theme  of  providing 
access  to  education  for  the  residents  of  the  state  of  California  but  rather  the  development  of  a nationally  focused 
revenue-generating  institution  that  would  be  the  “Amazon.com”  of  the  education  community.  CVU  was  forced  to 
rely  on  non-state  revenue  for  survival.  In  contrast,  the  surviving  public  virtual  universities  were  based  on  a central 
theme  of  providing  access  to  education  via  the  academic  and  student  services  components  of  the  institution.  These 
defining  elements  of  the  policies  of  CVU  appear  to  set  it  apart  from  the  surviving  public  virtual  universities. 

Implications  for  Practice 

Four  significant  implications  for  practice  emerged  from  this  study.  These  implications  for  practice  can  be 
applied  independently  or  as  a whole. 
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Implication  1 : Development  of  Faculty  Resources 

The  first  implication  for  practice  was  that  faculty  development  resources  and  faculty  training  opportunities 
are  an  integral  element  in  the  development  and  operation  of  a successful  comprehensive  public  virtual  university.  It 
seems  apparent  that  the  creation  of  these  types  of  resources  for  the  faculty  both  motivates  and  encourages  faculty 
members  to  participate  in  the  development  and  instruction  of  online  courses. 

Implication  2:  Student  Services 

The  second  implication  for  practice  is  that  comprehensive  student  services  that  support  the  academic 
offerings  of  an  institution  are  essential  in  the  development  of  a successful  public  virtual  university.  As  demonstrated 
by  the  institutions  included  in  this  study  that  provide  online  student  services,  comprehensive  student  services  enable 
a student  to  successfully  meet  academic  goals  in  an  online  environment.  In  addition,  online  student  services  that  are 
provided  in  a “one-stop  shop’’  method  seemingly  enhance  the  availability  of  these  resources  to  the  students.  It  is  a 
logical  expectation  that  virtual  universities  that  allow  a student  to  apply  online,  register  for  a class  online,  manage 
their  student  account,  or  process  their  financial  aid  application  in  a “one-stop  shop’’  environment  would  experience 
success  compared  to  their  counterparts. 

Implication  3:  Funding 

The  third  implication  for  practice  is  that  the  level  of  annual  funding  for  a public  virtual  university  may  be 
directly  related  to  the  success  of  a public  virtual  university.  This  implication  is  critical  for  stakeholders  in  public 
virtual  universities  that  are  experiencing  low  student  enrollments.  It  should  be  noted  that  this  finding  may  not  be 
isolated  in  that  it  was  directly  related  to  an  institution’s  ability  to  provide  faculty  development  resources  or 
comprehensive  student  services.  Therefore,  it  is  a logical  conclusion  that  a virtual  university  with  increased  levels 
of  funding  and  state-level  support  would  be  able  to  provide  these  additional  resources  for  students  and  faculty  and, 
therefore,  experience  significant  increases  in  annual  student  enrollment. 

Implication  4:  Access  to  Education 

The  fourth  and  final  implication  for  practice  is  that  each  successful  and  surviving  public  virtual  university 
maintained  a commitment  to  the  core  concept  of  providing  access  to  education  in  the  host  state  that  it  served.  These 
concepts  and  themes  included  access  to  technology,  access  to  Internet  resources,  scholarships,  policies  that  support 
learner-centered  instruction,  instruction  models  that  allow  students  to  work  in  small  groups,  and  curricula  organized 
around  central  concepts.  The  institutions  that  have  demonstrated  the  ability  to  survive  and  be  successful  have  also 
demonstrated  a commitment  to  one  or  more  of  these  themes  and  concepts.  The  general  idea  that  these  institutions 
were  designed  to  increase  access  to  education  is  seemingly  fundamental  to  the  success  of  a public  virtual  university. 

Advice  for  Policy  Makers 

The  findings  from  this  study  can  be  extrapolated  to  provide  guidelines  for  educational  and  legislative 
stakeholders  who  are  involved  in  the  development  or  operation  of  a public  virtual  university.  The  following  six 
recommendations  are  based  on  the  findings  from  this  study.  These  recommendations  are  presented  in  a manner  that 
is  applicable  for  the  educational  or  legislative  stakeholder  who  is  directly  involved  with  the  development  of  a new 
public  virtual  university.  The  recommendations  for  policy  makers  are  as  follows: 

1.  Develop  comprehensive  student  services  in  a “one-stop  shop’’  environment.  Students  should  be  able  to 
apply,  register,  make  tuition  payments,  and  complete  the  financial  aid  application  process  in  a virtual  student  center 
that  provides  all  of  the  same  resources  as  a traditional  student  center. 

2.  Develop  integrated  faculty  development  resources.  The  faculty  who  develop  and  teach  online  courses 
should  have  access  to  training  opportunities  and  instructional  technology  resources. 

3.  Maintain  an  institutional  focus  on  providing  increased  access  to  educational  resources.  This  theme 
should  be  present  in  institutional  goals  and  mission  statements. 

4.  Develop  a financial  plan  that  is  based  on  state-appropriated  hard  money.  Be  careful  of  private 
partnerships  and  sponsors  who  want  to  participate  in  the  operation  and  potential  revenues  from  the  institution.  If  a 
public  virtual  university  is  going  to  be  successful,  it  will  require  a strong  commitment  from  the  state  board  of 
regents  or  legislative  bodies. 

5.  Review  the  various  methods  for  the  classification  of  a virtual  university  and  develop  the  virtual 
university  with  the  intention  of  creating  an  institution  that  meets  the  needs  of  the  constituents  in  a manner  that  is 
consistent  with  the  type  of  virtual  institution  the  educational  stakeholders  require. 

6.  Carefully  project  the  annual  budget  for  the  virtual  university.  Remember  that  there  is  a potential 
relationship  between  the  level  of  annual  funding  and  the  success  of  the  institution.  Projecting  fiscal  needs 
adequately  can  have  a dramatic  impact  on  the  success  of  the  virtual  university. 


295 


References 

Altbach,  P.  G.,  & Berdahl,  R.  O.  (1981).  Higher  education  in  American  society  (rev.  ed.).  Buffalo,  NY: 
Prometheus  Books. 

Aoki,  K.,  & Pogroszewski,  D.  (1998).  Virtual  university  reference  model:  A guide  to  delivering  education 
and  support  services  to  the  distance  learner.  Online  Journal  of  Distance  Learning  Administration,  1 (3),  1-15. 

Arizona  Board  of  Regents.  (2001)  Homepage  [online].  Available:  http://ww.abor.asu.edu 

Arizona  Distance  Learning  Catalog  [online].  (2000).  Available:  http://azdistancleaming.org 

Ary,  D.,  Jacobs,  L.  C.,  & Razavieh,  A.  (2002).  Introduction  to  research  in  education  (6th  ed.).  Belmont, 

CA:  Wadsworth/Thomson  Learning. 

Baer,  W.  S.  (1998).  Will  the  Internet  transform  higher  education.  Annual  Review  of  Institute  for 
Information  Studies  [online].  Available:  http://www.rand.org/publications/RP/RP685.pdf 

Banerjee,  P.  (1999).  Assessing  the  effects  of  learning  styles  and  attitudes  toward  computers  on  users' 
evaluation  of  three  computer-assisted  instruction  modules  in  health  education.  Unpublished  doctoral  dissertation. 
Southern  Illinois  University,  Carbondale. 

Bardach,  E.  (2000).  A practical  guide  for  policy  analysis:  The  eightfold  path  to  more  effective  problem 
solving.  New  York:  Chatham  House  Publishers,  Seven  Bridges  Press. 

Berg,  G.  A.  (1998).  Public  policy  on  distance  learning  in  higher  education:  California  State  University  and 
Western  Governors  Association  initiatives.  Education  Policy  Analysis  Archives,  6(11)  [online].  Available: 
http://epaa.asu.edu/epaa 

Berge,  Z.  L.  (1998).  Barriers  to  online  teaching  in  postsecondary  institutions:  Can  policy  changes  fix  it? 
Online  Journal  of  Distance  Learning  Administration,  1(2)  [online].  Available:  http://www.westga.edu 

Brennan,  I .L.,  de  Vries,  P.,  & Williams,  R.  (1997).  Standards  and  quality  in  higher  education.  London: 
Jessica  Kingsley  Publishers. 

Brubacher,  J.  S.,  & Rudy,  W.  (1976).  Higher  education  in  transition:  A history  of  American  colleges  and 
universities,  1636-1976  (3rd  ed.).  New  York:  Harper  & Row. 

California  Virtual  Campus  [CVC].  (2000).  Homepage  [online].  Available:  http://www.cvc.edu 
Carnevale,  D.  (2000a,  August  1 1).  Accrediting  bodies  consider  new  standards  for  distance-education 
programs.  Chronicle  of  Higher  Education  [online].  Available:  http://chronicle.com 

Carnevale,  D.  (2000b,  September  12).  Arizona  regents  plan  an  online  institution  that  would  offer  degrees. 
Chronicle  of  Higher  Education  [online].  Available:  http://chronicle.com 

Carnevale,  D.  (2000c,  July  7,  2000).  Teachers  union  opposes  undergraduate  degrees  earned  wholly  at  a 
distance.  Chronicle  of  Higher  Education  [online].  Available:  http://chronicle.com 

Carnevale,  D.  (2000d,  May  19).  Western  Governors  U.  is  behind  projections  while  Southern  effort  exceeds 
expectations.  Chronicle  of  Higher  Education  [online].  Available:  http://chronicle.com 

Carnevale,  D.  (2001a,  January  5).  Commission  says  federal  rules  on  distance  education  must  be  updated. 
Chronicle  of  Higher  Education  [online].  Available:  http://chronicle.com 

Carnevale,  D.  (2001b,  January  17).  Teachers  union  urges  colleges  to  adopt  its  distance-education  standards. 
Chronicle  of  Higher  Education  [online].  Available:  http://chronicle.com 

Coleman,  J.  S.  (1998).  Equality  and  achievement  in  education.  San  Erancisco,  CA:  Westview  Press. 

Council  for  Higher  Education  Accreditation  [CHEA].  (1999).  Distance  learning  in  higher  education 
[online].  Available:  http://www.ihep.com 

Council  for  Higher  Education  Accreditation  [CHEA].  (2000).  Distance  learning  in  higher  education: 

CHEA  update  #3  [online].  Available:  http://www.ihep.com 

Crow,  S.  (2000).  Accreditation  of  online  institutions.  Technology  Source  [online].  Available: 
http://horizon.unc.edu/TS/vu 

Daniel,  J.  S.  (1996).  Mega-universities  and  knowledge  media:  Technology  strategies  for  higher  education. 
London:  Kogan  Page  Publishing. 

Darling-Hammond,  L.  (1997).  The  right  to  learn.  San  Erancisco,  CA:  Jossey-Bass. 

Duderstadt,  J.  J.  (1997).  The  future  of  the  university  in  an  age  of  knowledge.  JALN,  7(2),  78-88. 

Duin,  A.,  & Baer,  L.  (2000,  November).  Virtual  U:  Creating  the  Minnesota  Virtual  University— Assessing 
results  and  readiness  criteria.  Educause  Quarterly,  16-22. 

Durant,  W.,  & Durant,  A.  (1968).  The  lessons  of  history.  New  York:  Simon  and  Schuster. 


296 


Edwards,  N.  (1939).  Equal  educational  opportunity  for  youth:  A national  responsibility.  Washington,  DC: 
American  Council  on  Education. 

Enockson,  J.  O.  (1997).  An  assessment  of  an  emerging  technological  delivery  system  for  distance 
education.  Unpublished  doctoral  dissertation.  Northern  Arizona  University,  Elagstaff. 

Epper,  R.  M.  (1999).  State  policies  for  distance  education:  A survey  of  the  states  [online].  Available: 
http://www.sheeo.org 

Elorida  Virtual  Campus  [EVC].  (2000).  Homepage  [online].  Available:  http://flcampus.org 
Gellman-Danley,  B.,  & Eetzner,  M.  J.  (1998).  Asking  the  really  tough  questions:  Policy  issues  for  distance 
learning.  Online  Journal  of  Distance  Learning  Administration,  7(1)  [online].  Available:  http://www.westga.edu 
Georgia  Global  learning  online  for  business  and  education  [online].  (2000).  Available: 
http://georgiaglobe.org 

Gladieux,  L.  E.,  & Swail,  W.  S.  (1999).  The  virtual  university  and  educational  opportunity:  Issues  of  equity 
and  access  for  the  next  generation.  Washington,  DC:  The  College  Board. 

Glossary  of  distance  education  terminology  [online].  (2002).  Available:  www.uidaho.edu/evo/distl3.html 
Goodlad,  J.  I.  (1983).  A place  called  school.  Alexandria,  VA:  Association  for  Supervision  and  Curriculum 
Development. 

Goodlad,  J.  I.  (1988).  We  must  offer  equal  access  to  knowledge.  Educational  Leadership,  4(5), 16- 22. 
Goodlad,  J.  I.  (1990).  Common  schools  for  the  commonweal:  Reconciling  self-interest  with  the  common 
good.  Access  to  knowledge:  An  agenda  for  our  nation 's  schools.  New  York:  College  Board  Publications. 

Goodlad,  J.  L,  & Keating,  P.  ( 1983).  Access  to  knowledge.  Teachers  College  Record,  84{A),  787-800. 
Graves,  W.  H.  (1997).  Eree  trade  in  higher  education:  The  meta-university.  Journal  for  Asynchronous 
Learning  Networks,  7(1),  97-108. 

Green,  T.  E.  (1994).  Policy  questions:  A conceptual  study.  Education  Policy  Analysis  Archives,  2(7) 
[online].  Available:  http://epaa.asu.edu/epaa/v2n7.html 

Gunn,  C.  (2000,  July/ August).  Illinois  Virtual  Campus:  Eocusing  on  student  support.  Technology  Source 
[online].  Available:  http://horizon.unc.edu 

Harasim,  L.,  (1999).  A framework  for  online  learning:  The  virtual-U.  IEEE  Computer  Society  Journal, 
52(3),  44-49. 

Heterick,  R.  C.  (1995,  May  1).  The  virtual  university.  Educom  Review  [online].  Available: 
http://educom.edu 

Hettinger,  J.  (1997,  October).  Degree  by  mail.  Techniques  Magazine  [online].  Available: 
http://www.acteonline.org/public/techniques.html. 

Hodgson,  G.  (1973,  March).  Inequality:  Do  schools  make  a difference?  Atlantic  Monthly,  40-43. 

Horwitz,  J.  (2000,  July  6).  AET  proposes  standards  for  online  colleges  and  universities.  Chronicle  of 
Higher  Education  [online].  Available:  http://chronicle.com 

Hurst,  E.  (1999,  July/ August).  So  you  want  to  start  a virtual  university?  On  the  Horizon  [online].  Available: 
http  ://horizon.  unc  .edu 

Illinois  Virtual  Campus  [IVC].  (2000).  Homepage  [online].  Available:  http://ivc.illinois.edu 
Indiana  College  Network  [ICN].  (2000).  Homepage  [online].  Available:  http://www.icn.org/ 

Inglis,  A.  (1999).  Delivering  digitally:  Managing  the  transition  to  the  knowledge  media.  London:  Kogan 
Page  Limited. 

Irving,  C.  (1999,  Summer).  California  Virtual  University:  Highly  touted  experiment  falls  short  of 
expectations.  National  Crosstalk  [online].  Available:  http://www.highereducation.org/crosstalk 

Keating,  P.  (1990).  Striving  for  sex  equity  in  schools.  Access  to  knowledge:  An  agenda  for  our  nation’s 
schools.  New  York:  College  Board  Publications. 

Kentucky  Virtual  University  [KYVU].  (2000).  Homepage  [online].  Available:  http://www.kyvu.org 
Kerr,  C.  (1994).  Higher  education  cannot  escape  history:  Issues  for  the  21st  century.  Albany,  NY:  State 
University  of  New  York  Press. 

King,  J.  W.,  Nugent,  G.  C.,  Russell,  E.  B.,  Eich,  J.,  & Lacy,  D.  D.  (2000).  Policy  frameworks  for  distance 
education:  Implications  for  decision  makers.  Online  Journal  of  Distance  Learning  Administration,  5(2),  [online]. 
Available : http  J/www . westga.  edu/ 

Kulik,  J.  A.,  & Kulik,  C.  L.  (1991).  Effectiveness  of  computer-based  instruction:  An  updated  analysis.  Ann 
Tkrbor,  MI:  Center  for  Research  on  Learning  and  Teaching. 

MarylandOnline  [MOL].  (2000).  Homepage  [online].  Available:  http://marylandonline.org 
Michigan  Virtual  University  [MVU].  (2000).  Homepage  [online].  Available:  http://www.mivu.org/ 


297 


Minnesota  Virtual  University  [MNVU].  (2000).  Homepage  [online].  Available:  http://www.mnvu.org 
Mood,  A.  (1983).  Introduction  to  policy  analysis.  New  York:  Elsevier  Publishing. 

National  Education  Association.  (2000).  A survey  of  traditional  and  distance-learning  higher  education 
members  [online].  Available:  http://www.nea.. org 

New  Jersey  Virtual  University  [NJVU].  (2001).  Homepage  [online].  Available:  http://www.njvu.org 
Ohio  Learning  Network  [OLN].  (2000).  Homepage  [online].  Available:  http://oln.org/ 

Online  College  of  Oklahoma  [OCO].  (2000).  Homepage  [online].  Available: 
http://www.okcollegeonline.org 

Page,  D.  (2000).  KYVU:  Bringing  it  all  together.  Eduprise  [online].  Available: 
http://www.convergemag.com 

Persell,  C.  H.  (1977).  Education  and  inequality.  New  York:  Macmillian  Publishing  Co. 

Rosevear,  S.  G.  (1999a,  April).  Lessons  for  developing  a partnership -based  virtual  university.  Technology 
Source  [online].  Available:  http://horizon.unc.edu/  TS/vu 

Rosevear,  S.G.  (1999b).  Understanding  the  formation  of  a cross-industry  alliance  in  the  knowledge 
industry.  Unpublished  doctoral  dissertation.  University  of  Michigan,  Ann  Arbor. 

Rudolph,  E.  (1990).  The  American  college  and  university:  A history.  Athens,  GA:  University  of  Georgia 

Press. 

Russell,  T.  L.  (1999).  The  no  significant  difference  phenomenon  [online].  Chapel  Hill,  NC:  Office  of 
Instructional  Telecommunications,  North  Carolina  State  University.  Available: 
http://www.krwebdesign.com/data/nosigdif.htm 

Schutte,  J.  G.  (1996).  Virtual  teaching  in  higher  education:  The  new  intellectual  superhighway  or  just 
another  traffic  jam?  [online].  Northridge,  CA:  California  State  University.  Available: 
http  ://w  w w . csun.edu/sociology/virexp . htm 

Sherry,  A.  C.,  Eulford,  C.  P.,  & Zhang,  S.  (1998).  Assessing  distance  learners’  satisfaction  and  instruction: 
A quantitative  and  a qualitative  measure.  American  Journal  of  Distance  Education,  12(3),  4-28. 

Smith,  B.  (1998,  December).  Creating  consortia:  Export  the  best,  import  the  rest.  Converge  [online]. 
Available:  http://www.convergemag.com 

Smith,  P.  L.,  & Dillon,  C.  L.  (1999).  Comparing  distance  learning  and  classroom  learning:  Conceptual 
considerations.  American  Journal  of  Distance  Education,  13(2),  6-23. 

South  Dakota  Electronic  University  Consortium  [EUC].  (2000).  Homepage  [online].  Available: 
http://hpcnet.org 

Southern  Regional  Education  Board  [SREB].  (2000).  Homepage  [online].  Available: 
http :// WWW . srec . sreb . org 

Southern  Regional  Electronic  Campus  [SREC].  (2000).  Homepage  [online].  Available: 
http://www.electroniccampus.org 

State  University  of  New  York  (SUNY)  Learning  Network  [SLN].  (2000).  Homepage  [online].  Available: 
http://sln.suny.edu/ 

Teare,  R.,  Davies,  D.,  & Sandelands,  E.  (1999).  The  virtual  university:  An  action  paradigm  and  process  for 
workplace  learning.  New  York:  Wellington  House. 

Tennessee  Board  of  Regents.  (2000).  Homepage  [online].  Available:  http://tbr.state.tn.us 
The,  G.  P.  L.,  & Eraser  B.  J.  (1994).  An  evaluation  of  computer-assisted  learning  in  terms  of  achievement, 
attitudes,  and  classroom  environment.  Evaluation  and  Research  in  Education,  S(3),  147-162. 

Thompson,  J.  (1996,  November).  The  California  Virtual  University.  Paper  presented  at  the  11*  Annual 
DET/CHE  Conference,  Monterey,  CA  [online].  Available:  http:I- 
dwww.ucsb.edu/detche/library/distance/calif/calvu.html 

Thrail,  T.  A.  (1999,  March).  Welcome  to  the  virtual  university.  Technology  Source  [online].  Available: 
http://horizon.unc.edu/TS/vu 

Twigg  C.  A.  (1994).  The  need  for  a national  learning  infrastructure.  Educause  Review,  29(4,  5,  6)  [online]. 
Available:  http  J/www . educause. edu 

Twigg,  C.  A.  (1998).  The  Pew  learning  and  technology  program.  Pew  Grant  Program  in  Course  Redesign, 
Readiness  Criteria  [online].  Available:  http://center.rpi.edu/pewGrRdi.html 

Twigg,  C.  A.,  & Oblinger,  D.  (1996,  November  5).  The  virtual  university  [online].  Washington,  DC: 
Educam/IBM.  Available:  http://www.educause.edu 

University  of  Idaho,  College  of  Engineering.  (2002).  Glossary  of  distance  education  terminology  [online]. 
Available:  http://www.idaho.edu/evo/distglan.html 


298 


University  of  Texas  [UT]  TeleCampus.  (2000).  Homepage  [online].  Available: 
http  ://www . uol . com/telecampus 

Virtual  revolution  in  teaching.  (2000,  March  3).  Los  Angeles  Times,  Sec.  3.,  7. 

Webopedia  [online].  (2002).  Available:  http://www.pcwebopedia.com 

Weimer,  D.  L.,  & Vining,  A.  R.  (1999).  Policy  analysis:  Concepts  and  practice  (3‘^‘*  ed.).  Upper  Saddle 
River,  NJ:  Prentice-Hall  Inc. 

Western  Governors  University  [WGU].  (2000).  Homepage  [online].  Available:  http://www.wgu.edu 

Wisconsin  learning  innovations  [online]  (2000).  Available:  http://learn.  wisconsin.edu/ 

Witt,  A.  A.,  Wattenbarger,  J.  L.,  Gollattscheck,  J.  F.,  & Suppiger,  J.  E.  (1994).  America’s  community 
colleges:  The  first  century.  Washington,  DC:  Community  College  Press,  American  Association  of  Community 
Colleges. 

Wolf,  D.  B.,  & Johnstone,  S.  M.  (1999).  Cleaning  up  the  language  of  electronically  delivered  academic 
programs.  Change  [online].  Available:  http://www.wiche.edu 

Young,  J.  (2000,  June  30).  Veteran  of  California  Virtual  U.  blames  flawed  business  plan  for  its  demise. 
Chronicle  of  Higher  Education  [online].  Available:  http://chronicle.com 


299 


Creating  an  Organic  Knowledge -Building  Environment  Within  an 
Asynchrous  Distributed  Learning  Context 

Leslie  Moller 
University  of  North  Texas 

Gustavo  E.  Prestera 
Pennsylvania  State  University 

Douglas  Harvey 

Richard  Stockton  College  of  New  Jersey 

Margaret  Downs -Keller 
Consultant 

Jo- Ann  McCausland 
Galaxy  Scientific  Corporation 


Organic  Architecture 

In  architecture,  “organic”  refers  to  a philosophical  approach  that  promotes  structures  for  living  that  are 
harmonious  to  the  external  environment  with  an  emphasis  on  spontaneity  (Satler,  1999).  The  goal  of  organic 
architecture,  according  to  architect  Frank  Lloyd  Wright,  was  to  “create  a building  that  did  more  than  enclose  or 
close  off  possibilities.”  As  Satler  (1999)  has  noted,  Wright  “hoped  to  create  works  that  sheltered  but  also 
encouraged  movement,  beyond  what  apparently  existed”  (p.  54).  Wright’s  works  were  based  on  the  destruction  of 
the  “box”  that  separated  the  individual  from  the  world.  Organic  buildings  attempted  to  create  a harmonious  and 
seamless  environment  where  people  were  protected  yet  not  isolated  from  life  outside  the  walls.  By  moving  people 
from  structures  that  created  a sense  of  isolation  to  structures  that  blended  into  the  environment,  a different  type  of 
relationship  could  develop  between  an  individual  and  the  larger  whole.  No  longer  would  an  individual  (structure) 
remain  in  the  background  (subordinate  to  a larger  whole)”  (Satler,  1999  p.  54).  Referring  to  Wright,  Satler  wrote, 
“Psychologically,  this  paved  the  way  towards  a view  of  architecture,  and  more  fundamentally,  a view  of  interaction, 
that  respected  and  responded  to  both  individual  and  collective  needs  (p.  52).” 

This  article  suggests  that  instructional  designers  should  consider  Wright’s  organic  philosophy  as  they 
design  and  build  asynchronous  distributed  learning  environments  so  that  a more  natural  learning  space  is  created 
that  fosters  knowledge-building.  In  designing  such  environments,  the  goal  is  to  foster  the  building  of  knowledge  by 
removing  the  barriers  that  separate  people  from  their  environment.  Traditional  instruction,  much  like  traditional 
architecture,  imposes  a structure  on  its  environment.  While  some  may  argue  this  is  necessary  to  control  the  learning 
process,  it  has  resulted  in  obstacles  that  close  the  individual  off  from  its  human  nature  to  learn.  Constructivists 
assert  that  while  most  traditional  instruction  does  well  to  control  and  manage  the  educational  experience,  it  does 
little  to  maximize,  and  may  even  inhibit,  natural  human  learning  abilities  (Marshall,  1997).  Learners  need  an 
environment  that  promotes  the  types  of  transformative  or  generative  processes  needed  to  handle  ill -structured 
situations,  including  thinking,  creativity,  collaboration,  dialogue,  and  argumentation  directed  at  solving  those 
problems  and  providing  continuous  learning  (Preskill  & Torres,  2000). 

Too  often,  traditional  instruction  and  virtual  classroom  forms  of  distance  learning  do  much  the  same: 
create  barriers  to  the  individual’s  natural  learning  inclinations.  What  is  needed  are  models,  which  like  Wright’s 
architectural  designs,  that  break  from  tradition  to  work  with,  rather  than  against,  the  natural  learning  instincts  and 
abilities  of  human  beings.  One  potential  conceptual  model  is  organic  knowledge-building.  This  model  leverages 
the  technological  capacity  of  asynchronous  distributed  learning  (ADL)  by  potentially  creating  environments  that 
allow  for  instructional  models  that  are  impractical  or  inefficient  in  a traditional  classroom. 


Defining  Organic  Knowledge -Building 

This  article  argues  that  learning  environments  should  be  designed  and  constructed  using  an  organic 
approach,  so  that  learning  is  not  viewed  as  a distinct  human  activity  but  incorporated  into  everyday  performance. 


300 


Learning  environments  developed  from  an  organic  perspective  help  individuals  construct  knowledge  by  making 
available  to  them  a wide  variety  of  resources,  experiences,  and  interactions  to  explore.  As  such,  to  make 
knowledge-building  organic,  learning  is  defined  as  a constant  dynamic  and  purposive  process  in  which  cognitive 
capabilities  are  used  to  create  new  knowledge,  as  applications  or  solutions  to  problems,  by  first  reexamining  current 
knowledge  in  terms  of  a deeper  understanding  of  the  complexity  of  the  information;  second,  by  identifying  novel 
relationships  between  previously -believed  distinct  knowledge;  and,  finally,  by  identifying  unknown  but  required 
new  information.  Note,  knowledge  and  information  are  not  interchangeable  terms.  Information  refers  to  facts, 
concepts,  rules,  etc.,  that  are  somewhat  objective  and  easily  transferable.  Information  becomes  knowledge  when  an 
individual  takes  ownership  and  makes  it  meaningful  by  applying  or  using  the  information  in  conjunction  with  other 
established  knowledge. 

Organic  Learning  Environments 

To  support  the  previously  mentioned  definition,  organic  knowledge-building  substitutes  a highly 
asynchronous,  learner-centered,  technology-rich,  competency-based  environment  for  a bureaucratic  or  highly 
structured  classroom  model.  This  conceptual  learning  model  strives  to  place  the  learner  within  a “space”  in  which 
learning,  context,  and  application  can  co-exist  (Barab  & Duffy,  2000).  Land  and  Hannafin  (2000)  suggest  that  such 
spaces  are  more  learner-centered  than  traditional  computer-based  instruction.  Computer  or  web-based  ADLs  can 
provide  such  spaces  since  they  have  mechanisms  for  easing  and  thus  increasing  the  communication  among  learners, 
including  reconceptualizing  learning  from  one-shot  fixed  term  to  an  on-going  process  that  is  intermingled  with  the 
actual  work  processes. 

Learning  occurs  naturally  and  embodies  many  forms,  sometimes  taking  place  subconsciously  and 
serendipitously,  while  at  other  times  planned  and  deliberate.  We  can  learn  from  reading  a newspaper,  speaking  to  a 
colleague,  experiencing  something  new,  or  even  by  overhearing  a conversation  on  the  bus.  At  work  or  everyday 
life,  individuals  leverage  a wide  variety  of  resources  and  experiences  to  learn,  outside  the  boundaries  of  formal 
education  or  training,  about  the  things  that  matter  to  them.  They  commonly  experiment  with  ideas,  share  their  own 
knowledge  with  others,  collect  feedback,  and  assimilate  or  reject  the  knowledge  of  others.  Similarly,  learning 
environments  developed  from  an  organic  perspective  help  individuals  construct  knowledge  by  making  available  to 
them  a wide  variety  of  resources,  experiences,  and  interactions  to  explore. 

In  recommending  an  organic  philosophical  approach,  we  are  promoting  the  concept  that  instruction  should 
adapt  to  how  people  naturally  learn,  with  the  individual,  rather  than  the  group,  at  the  center.  Taking  the  organic 
concept  further,  we  support  a situated  cognitive  approach  and  argue  that  learning  is  not  isolated  from  doing  and  that 
instruction  should  serve  to  encourage  exploration,  experimentation,  communication,  and  growth.  Proponents  of 
situated  cognition  argue  that  learning  takes  place  during  the  individual’s  struggle  to  solve  a contextually  meaningful 
problem  (Brown,  Collins,  & Duguid,  1996;  Lave  & Wenger,  1991).  We  are  also  mindful  that  social  constructivists, 
on  the  other  hand,  suggest  that  learning  takes  place  in  the  process  of  communicating  ideas  and  negotiating  group 
meaning  (Jonassen,  1999;  Scardamalia  & Bereiter,  1993). 

To  the  question  of  whether  learning  takes  place  individually  or  in  groups,  the  Organic  Knowledge-Building 
model  answers  “yes.”  We  believe  that  learning  takes  place  within  the  individual  as  he  or  she  engages  in  a process  of 
solving  meaningful  problems  and  is  further  encouraged  and  supported  by  the  group  as  it  moves  closer  to  a shared 
meaning  (Brown,  2000).  This  edges  close  to  what  Torraco  (1999)  describes  as  skill.  “Skill  is  not  defined 
exclusively  in  terms  of  human  capabilities;  it  retains  features  of  both  the  performer  and  the  work  environment  in 
which  it  is  expressed”  (p.  256).  However,  Torraco  (1999)  also  states  that  “Skilled  performance  in  many  work 
environments  is  a function  of  expertise  that  is  distributed  among  group  members.  ...and  that  learning  is  a group 
activity”  (p.  254).  Clearly,  the  individual  who  is  allowed  to  develop  to  the  maximum  extent  is  a greater  contributor 
to  a larger  collective  learning  which  in  turn  nurtures  the  individual. 

Components  of  the  Organic  Knowledge -Building  Model 

In  organic  knowledge-building,  an  instructional  environment  is  created  that  utilizes  the  naturalistic, 
spontaneous,  and  regenerative  nature  of  humans  as  it  relates  to  learning.  Although  this  is  not  a new  concept,  per  se, 
we  believe  this  model  describes  a design  structure  that  can  aid  implementation. 

There  are  four  components  in  an  organic  knowledge-building  environment:  information  objects,  scaffolds, 
discourse  action  communities,  and  facilitation.  The  components  are  neither  sequential  nor  discrete;  their 
relationship  can  be  characterized  as  inter-related,  dynamic,  idiosyncratic,  often  overlapping.  In  short,  what  makes 
the  instruction  organic  is  not  any  particular  component,  but  rather  the  idea  that  these  components  are  used  in  concert 
with  the  individual’s  context  to  create  a rich  and  open  environment  that  is  alive  with  opportunities  for  exploration 
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and  meaning- making.  Furthermore,  this  model  is  not  bound  by  traditional  theoretical  constraints  but  adheres  to  a 
pragmatic  orientation  that  suggests  incorporating  any  “tool”  or  idea  that  helps  the  learner  to  accomplish  the  goal. 


FIGURE  1:  Visual  depiction  of  the  Organic  Knowledge-Building  Model 
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Figure  1 depicts  the  major  characteristics  of  the  organic  knowledge-building  model.  The  individual  is 
central  to  the  process  and,  simultaneously,  both  a learner  and  practitioner.  The  learner  is  driven  by  a meaningful 
problem  within  a workplace  or  other  reaHife  context.  The  learning  environment  is  what  we  traditionally  think  of  as 
the  conceptual  boundaries  of  a course,  curriculum,  training  program,  etc.  Note  that  the  boundaries  within  this  model 
are  not  solid.  In  an  organic  learning  environment,  there  should  indeed  be  significant  overlap  between  the  learning 
environment,  context,  problem,  and  learner.  In  organic  architecture,  structures  are  designed  to  blend  in  with  the 
natural  setting,  blurring  the  lines  between  the  home  and  its  natural  surroundings.  Similarly,  organic  learning 
environments  blur  the  distinction  between  the  individual  as  a learner  and  the  individual  as  a practitioner;  between 
the  learning  context  (e.g.,  the  classroom)  and  the  performance  context  (e.g.,  the  workplace). 

Another  aspect  of  this  model  is  the  availability  of  resources  that  encourage  and  support  the  individual  in 
exploring,  experimenting,  and  knowledge-building.  There  are  four  general  categories  of  resources:  information 
objects  (lOs),  scaffolds,  discourse  action  communities  (DACs),  and  facilitation.  Information  objects  are  essentially 
building  blocks  of  content.  They  can  stand  alone  or  they  can  be  strung  together  to  form  larger  molecules.  In  a 
traditional  sense,  lOs  replace  the  lectures,  handouts,  and  slides  given  to  students;  they  are  basically,  the  declarative 
(what)  and  procedural  (how)  knowledge  components  of  instruction.  Within  an  organic  learning  environment, 
individuals  have  access  to  a wide  variety  of  lOs,  representing  not  only  the  scope  of  one  expert’s  knowledge  and 
experience,  but  rather  the  entire  available  breadth  and  depth  of  multiple  knowledge  domains.  Discourse  action 
communities  do  not  differentiate  between  learning  communities  or  communities  of  practice.  Learners,  in  their 
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quest,  communicate  with  other  learners  and  practitioners  sharing  their  experiences  as  well  as  the  information  objects 
they  have  encountered  during  their  exploration.  These  interactions  in  turn  lead  to  the  discovery  of  other  information 
objects  within  the  learning  environment  as  well  as  outside  of  the  learning  environment.  From  this  ongoing 
exploration  and  discourse,  the  individuals  form  ideas,  what  we  may  call  hypotheses,  regarding  the  ill-structured 
problems  they  are  trying  to  solve.  Scaffolds,  some  in  the  form  of  simulators,  enable  the  learners  to  experiment  with 
their  ideas  within  a safe  environment  that  is  closely  related  to  their  context.  These  experimental  or  virtual 
workspaces  allow  us  to  eliminate  the  traditional  concept  of  practice.  Other  types  of  scaffolds  include  subject-matter 
experts,  exemplars/non-exemplars,  cases,  and  job  aids.  Together,  scaffolds,  information  objects,  and  discourse 
action  communities  create  a rich  environment  in  which  the  learner/practitioner  can  expbre,  interact,  and  construct 
meaning.  In  fact,  this  environment  is  so  rich  and  the  possible  paths  through  it  so  limitless  that  learners/practitioners 
may  be  overwhelmed  by  the  possibilities.  For  this  reason,  facilitation  is  critical  within  an  organic  learning 
environment  and  should  be  the  instructor’s  primary  role  in  the  learning  process. 

Visual  2 & 3 Samples  w/content  domain  examples 
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Knowledge  Objects 

• Provide  information 
Ergonomics 


o 

o 


Physical 
Environmental 
Social  aspects 


Scaffolds 


Ergonomic  checklists 
Models  produced  by 
companies  who  promote 
themselves  as  being 
ergonomically  correct 
3D  engineering 


A 


A 


Facilitator 

• Assist  in  question  and  answers 
with  students 

• Encourage  students  to  look 
deeper  into  their  research 


Learning  Communities 

• Students  will  share  their  research  findings  with  other 
groups  of  students  in  class 

• Discuss  the  ergonomic  features  of  the  classroom  and  home 
work  areas 

o Compare  the  ergonomic  features  found  similar 
o Compare  the  ergonomic  features  found  different 

• Construct  a new  checklist  or  add  to  the  existing  checklist 
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Knowledge  Objects 

• Information  on  different  products/services  offered  within  the 
organization 

• Common  objections  with  possible  responses 

• Information  on  the  consultative  sales  process  (general) 

• Information  on  the  most  difficult  aspects  of  the  sales  process 
(resolving  objections)  and  tips  to  maximize  your  efficiency 

• Information  on  tracking  reports  used  by  the  organization 
(sales  metrics)  including  how  to  complete  them 


caffolds 

• FAQs 

• Job-aid  including 
features/benefits  for  each 
product/service 

• Information  on 
competitors’ 
products/services 

• Examples  of  completed 
reports  (expense  reports, 
booking  reports, 
forecasts,  etc.) 

• Expert  salespeople 
within  the  organization 
talking  @ best  practices 

• Video  clips  of  good/bad 
sales  calls 


A 


V 


A 


V 


<A> 


Facilitation 

• Provide  case  studies 
for  salespeople  to 
recommend  as 
appropriate  product 
service 

• Role  plays  of 
possible  customers 


Learning  Communities 

• Discuss  case  studies  and  brainstorm  the  best  recommendation  for  a 
product/service 

• Salespeople  involved  in  an  online  chat  environment  to  talk  @ the 
challenges  they  face/offer  advice  to  each  other 

• Review  metrics  and  plan  how  to  do  better 

• Review/discuss  what  was  good/bad 


Information  Objects 

Information  objects  utilize  web  or  computer-based  tutorials  or  information  packets  to  convey  topics  or 
small  modules  of  the  key  portions  of  the  content.  Rather  than  a regularly  scheduled  and  managed  class  of 
determined  hours  where  learners  accumulate  information  to  be  used  or  examined  at  a later  time,  learners  directly 
engage  the  declarative  and  procedural  knowledge  in  smaller  chunks  at  a time  and  place  the  learner(s)  recognizes  it 
as  needed  information,  which  in  itself  is  a learning  activity. 

Information  objects,  or  “molecules”  of  information,  are  identified  and  categorized,  placed  in  a system 
where  the  information  can  be  quickly  extrapolated  and  applied  within  a new  frame  of  reference.  When  placed  into  a 
generic  categorized  framework,  learners  may  access  these  objects  by  using  key  words  to  retrieve  the  pertinent 
information.  Picture  these  “info  nuggets”  as  pamphlets  or  chunks  of  information  that  contain  specific  components 
of  a complete  procedure  or  process.  Learners  access  only  the  “info  nugget”  they  need  to  review,  rather  than 
spending  the  time  reviewing  the  full  procedure. 

For  example,  if  someone  wanted  to  learn  how  to  change  an  oil  filter,  he  or  she  could  access  the 
information  object  base  for  that  particular  chunk  of  information  and  it  might  show  a short  video  demonstrating  the 
procedure,  step-by-step.  But,  if  the  objective  were,  instead,  to  understand  car  systems  and  the  rules  associated  with 
them,  the  learner  would  need  more  structure,  organization,  and  content.  The  chunks  of  information  gathered  would 
be  more  detailed.  However,  when  the  learner  reached  the  part  about  learning  to  change  the  oil  filter,  the  same  video 
clip  would  be  part  of  the  program  (Gordon,  1997). 

Using  information  objects  supports  an  organic  knowledge-building  environment  by  requiring  the  learner  to 
navigate  the  learning  process  through  drawing  on  previously  formed  schemata,  incorporating  new  information  into 
them  as  needed  to  solve  a given  problem  or  enhance  dialogue  within  their  communities.  The  learning  object  format 
is  supported  by  the  belief  that  instructional  designers  can  create  independent  chunks  of  learning  content  that  provide 
a learning  experience  for  pedagogical  purposes.  The  chunks  are  self-contained,  though  they  also  may  contain 
references  to  other  objects;  they  may  be  combined  or  sequenced  to  form  longer  learning  interactions  (Quinn,  2000). 

This  supports  meaningful  and  useful  learning  by  allowing  the  learners  to  actively  create  meaning  by 
exploring,  experimenting,  testing,  and  applying  knowledge  in  self -directed  and  collaborative  ways,  rather  than 
following  a predetermined  format.  A problem  is  presented  to  learners  and  they  are  given  tools,  such  as  case  studies, 
information  resources,  or  collaboration  tools,  with  which  to  manipulate  the  information.  The  learner  then 
approaches  the  problem  at  his  or  her  own  pace  and  from  his  or  her  chosen  direction,  attempting  to  solve  the  problem 
with  the  tools  given  (Jonassen,  1999). 

The  use  of  information  objects  allows  the  learner  to  explore  the  content  in  unconventional  ways,  following 
a unique  path  to  discovering  the  targeted  learning  objective.  In  this  particular  application,  context  is  not  just 
provided  to  the  learner,  but  rather  provides  cues  to  the  learner  so  that  he  or  she  may  explore  the  content  in  linked 
contexts  and  draw  individual  conclusions  and  meanings.  The  use  of  the  Web  to  explore  the  content  can  be  enhanced 
by  the  use  of  knowledge  object  databases  to  focus  the  search  more  directly,  using  the  terminology  and  clues  that  the 
learner  uncovers  during  the  search  process. 

Developing  the  information  objects  involves  choosing  keywords  for  easy  and  broad  retrieval,  and  the 
avoidance  of  an  editorial  voice  in  the  information  packets,  which  might  include  introductions  or  conclusions  about 
the  content.  Additionally,  they  should  be  freestanding  or  modular,  non-sequential,  and  able  to  satisfy  a single 
learning  object.  To  assist  in  that  process,  the  system  should  use  “context  wrappers.”  Context  wrappers  consist  of 
information  that  is  associated  with  a learning  object.  One  object  may  have  multiple  wrappers,  providing  different 
ways  to  contextualize  it.  In  this  way,  the  keywords  would  allow  multiple  uses  and  frameworks  to  capture  the  same 
information  nuggets. 

Algorithms  can  be  used  to  guide  the  manipulation  of  knowledge  objects  into  different  categories  and 
permutations,  somewhat  like  a formula  manipulates  numbers  in  different  sequences  to  arrive  at  different  answers. 
Merrill’ s theory  supports  our  knowledge-building  approach  by  allowing  the  learner  to  control  the  path  for  his  or  her 
own  learning,  through  choices  in  the  program  (Merrill,  1999). 

The  advantages  of  using  knowledge  objects  as  a tool  for  creating  just-in-time  training  are  numerous,  but  the 
need  for  a consistent  and  regulated  information  structure  is  necessary  before  this  approach  can  be  used  globally. 
Gordon  (1997)  quotes  Matlack,  senior  developer  for  Multimedia  Pathways,  Inc.,  regarding  one  of  the  concerns  about 
chunking  bits  of  information: 

Taking  pieces  of  information,  bit-by-bit,  and  pasting  it  together  is  not  what  training  is  all  about.  Training  is 

about  taking  raw  informatbn,  integrating  it,  and  helping  you  develop  a mental  model  of  it.  In  the  end,  you 

should  be  synthesizing  all  this  information  and  coming  out  with  a transfer  of  know-how  rather  than  a 

transfer  of  facts,  (p.  32) 
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Scaffolding 

Technology-based  scaffolding  involves  helping  learners  by  providing  support  for  performance  when  they 
encounter  a task  or  concept  they  have  yet  to  perform  (Jonassen,  1996).  In  this  manner,  learners  are  able  to  find  help 
when  they  encounter  something  for  the  first  time.  This  approach  allows  learners  to  ask  questions,  see  examples  or 
models  of  the  process,  or  get  expert  advice  on  what  to  do  next.  As  with  other  parts  of  our  model,  learners  could  use 
these  tools  while  they  are  performing  a new  task. 

An  excellent  example  of  the  power  of  this  type  of  computer-based  scaffolding  can  be  seen  in  several 
products  developed  by  Schank  (Schank  & Cleary,  1995).  Schank  has  developed  a number  of  systems  he  dubs 
"ASK"  systems.  These  ASK  systems  allow  learners  to  ask  questions  using  the  program  and  have  their  questions 
answered  in  the  form  of  video  clips  of  experts  in  the  field  speaking  about  the  topic  at  hand.  For  example,  one  such 
system  is  TransASK,  developed  for  use  by  the  United  States  military  in  training  officers  in  materials  management 
during  a military  crisis  (Schank  & Cleary,  1995).  In  TransASK,  a trainee  can  input  questions  as  he  or  she  works 
through  either  a training  scenario  or  an  actual  crisis,  and  the  system  will  provide  a video  clip  from  among  a database 
of  clips  containing  25  experienced  officers  telling  their  stories  of  how  they  handled  a similar  situation.  While  ASK 
systems  are  highly  advanced  CBT  programs,  even  simple  tutorials  could  provide  the  information  and  modeling  a 
learner  needs,  at  the  time  when  the  learner/practitioner  needs  to  perform  a specific  skill.  Furthermore,  learning  new 
things  at  the  time  they  are  needed  helps  learners  make  the  knowledge  more  meaningful  and  stable  in  the  learner’s 
cognitive  processes.  This  also  allows  for  the  learning  process  to  continue  unabated. 

Scaffolding  also  should  incorporate  the  use  of  virtual  workspaces  and  simulators.  Unlike  a predetermined 
scenario,  as  typically  found  in  simulations,  this  virtual  laboratory  would  allow  the  learners  to  practice,  experiment, 
or  test  various  hypotheses,  actions,  and  models  in  relative  safety  while  still  maintaining  the  realistic  outcomes 
necessary  in  evaluating  and  revising  knowledge.  Examples  of  these  types  of  scaffolding  tools  range  from  the 
Electronic  Workbench  designed  to  allow  budding  chemists  to  try  out  electronic  circuits  (Sadik  & Cheung,  2001)  to  a 
multimedia  simulation  of  an  actual  work  environment  (Montgomery,  Campbell,  & Moffett,  1994). 

Eor  example,  the  multimedia  simulation  is  a goal  based  scenario  (GBS)  system,  a term  conceived  by 
Schank  (Schank  & Cleary,  1995)  and  similar  to  the  previously  described  ASK  systems.  GBS  systems  allow  learners 
to  simulate  the  work  environment  and  receive  immediate  feedback  on  their  performance.  Successful 
implementations  of  these  GBS  systems  can  allow  for  scaffolding  strategies  such  as  coaching  and  feedback  to  be 
used  within  a simulated  environment  (Acovelli  & Gamble,  1997). 

Discourse  Action  Communities 

One  of  the  key  principles  of  organic  environments  is  that  the  learner  should  neither  be  isolated  from  nor 
become  subsumed  by  the  larger  group.  In  an  asynchronous  organic  community,  a group  of  people  are  electronically 
linked  for  the  purpose  of  learning  specific  materials  through  collaboration  via  computer- mediated  communication 
(CMC).  These  groups  of  learners  are  typically  comprised  of  computer-literate  adults  who  are  seeking  to  gain 
knowledge  in  a particular  area  of  interest  and  are  highly  motivated  to  participate  in  all  the  learning  requirements 
(Downs  & Moller,  1999;  Garrison  1989;  Moore  & Kearsley  1996). 

Viewing  learning  as  primarily  a knowledge-building  process  requires  the  learner  to  be  actively  engaged  in 
cognitive  manipulation  of  the  instructional  content  or  information.  As  Moore  and  Kearsley  (1996)  point  out,  “learner 
to  learner  interaction  is  desirable  for  pedagogical  reasons.”  Gay  and  Lentini  (1995)  confirm  this  assumption  by 
writing  “learning  is  fundamentally  built  up  through  conversations  between  persons  or  groups,  involving  the  creation 
and  interpretation  of  communication”  (p.2).  Eurthermore,  they  argue,  “conversations  are  the  means  by  which  people 
collaboratively  construct  beliefs  and  meanings  as  well  as  state  their  differences”  (p.2).  Thus,  we  are  more  able  to 
enlarge  our  own  beliefs  and  more  likely  to  take  risks  when  supported  by  a community  of  other  learners  (Grabinger, 
1996). 

In  other  words,  an  organic  knowledge-building  community  is  designed  to  facilitate  learner-centered 
communication  and  increase  peer  interaction  by  first  engaging  the  learner  in  tasks  that  require  and  allow  for  high- 
level  thinking  and  reflection.  These  higher-order  cognitive  processes  are  prompted  through  the  use  of 
communication  for  collaborative  idea  generation,  critical  analysis,  and  argument  construction.  This  organic 
knowledge-building  discourse  and  action  community  provides  the  necessary  intellectual  and  emotional  support  to 
the  learner  for  growth  of  intellectual  risk-taking  behaviors. 

Online  learning  is  potentially  a powerful  tool  when  a community  encourages  learners  to  deeper  cognitive 
experiences  through:  collaboration,  debate/argument,  and  conflict/resolution.  Collaboration  is  the  most  noted  of 
these  interactions.  In  1988,  Douglas  Englebart  at  the  Augmented  Human  Intellect  Research  Center  of  Stanford 
Research  Institute  developed  AMgment.  This  online  learning  system  provides  tools,  like  chat  rooms,  to  support 
collaborative  knowledge  work  and  places  “the  greatest  emphasis  on  collaboration  among  people  doing  their  work  in 


307 


an  asynchronous,  geographically  distributed  manner”  (Palloff  & Pratt,  1999,  p.  245).  Traditional  distance  learning 
models  emphasize  the  independence  of  the  learner  (Downs  & Moller,  1999;  Moore,  1986)  and  the  privatization  of 
learning  (Keegan,  1986).  Organic  learning,  on  the  other  hand,  is  distinguished  by  the  social  nature  of  the  learning 
environment  that  is  created  through  community  and  collaborative  learning  (Harasim,  1990).  The  process  of 
collaborating  through  learning  communities  increases  engagement  in  the  learning  process.  When  promoted, 
“collaborative  work  forms  the  basis  for  the  student’s  ability  to  engage  in  a transformative  learning  process”  (Palloff 
& Pratt,  1999,  p.  127).  These  transformative  learning  processes  should  include  online  conflicts  among  participants. 

Controversy  and  conflict  within  a group  can  lead  members  to  question  their  own  concepts  as  well  as  seek 
new  information  and  perspectives  (Harasim,  1990).  Conflict  can  facilitate  learning  and  personal  growth  when  it  is 
used  with  the  goal  of  reaching  a deeper  level  of  understanding  (Palloff  & Pratt,  1999).  Additionally,  Johnson  and 
Johnson  (1979)  propose  that  conceptual  conflict  resolution  has  cognitive  benefits:  “Students  who  experience 
conceptual  conflict  resulting  from  controversy  are  better  able  to  generalize  the  principles  they  learn  to  a wider 
variety  of  situations  than  are  students  who  do  not  experience  such  conceptual  conflict”  (p.  67). 

One  of  the  concerns  about  conflict  and  resolving  conflict  in  online  learning  is  that,  without  the  physical  and 
verbal  cues  of  face-to-face  contact,  many  people  feel  less  socially  constrained  when  debating  an  argument.  When 
communicating  face-to-face,  conflicts  are  resolved  with  what  we  consider  to  be  socially  appropriate  behavior. 
(Palloff  & Pratt,  1999).  With  a defined  norm  for  how  to  deal  with  online  conflict,  a deeper  and  meaningful  type  of 
learning  can  take  place  (Harasim,  1990;  Moller,  Harvey,  Downs,  & Godshalk,  2000).  Considering  the  many  ways  in 
which  the  Internet  and  online  learning  touch  people’s  lives,  these  issues  deserve  further  discussion. 

Given  that  communities  are  formed  around  issues  of  identity  and  shared  values  rather  than  the  traditional 
place-  and  time  -based  development  of  community,  online  learning  provides  a platform  for  a new,  redefined  sense  of 
community  (Palloff,  1996).  An  organic  knowledge-building  community,  via  online  learning,  facilitates  learner- 
centered  communication  and  increases  peer  interaction  by  first  engaging  the  learner  in  a manner  that  is  more  active 
than  is  likely  to  be  found  in  traditional  learning;  promotes  an  opportunity  for  meaningful  learning  that  requires  and 
allows  for  thinking  and  reflection;  increases  cognitive  development  through  argument  construction,  communication 
of  those  ideas,  and  critical  analysis  of  new  ideas;  expands  the  range  of  ideas  and  capitalizes  on  the  possibilities  of 
brainstorming  or  collaborative  idea  generation;  and  provides  emotional  support  for  growth  or  intellectual  risk-taking 
behaviors. 

Community  begins  to  shape  as  the  learners  communicate,  negotiate,  and  work  with  one  another  toward  a 
common  goal.  In  a traditional  classroom,  this  sense  of  working  toward  a common  goal  and  building  a sense  of 
community  identity  often  does  not  occur.  Bielaczyc  and  Collins  (1998)  support  learning  communities  by  stating  that 
“. . .in  learning  communities,  both  the  content  learned  and  the  processes  of  learning  from  outside  resources  are 
shared  more  among  members  of  the  community  and  become  part  of  the  collective  understanding.  A further 
distinction  between  learning  communities  and  most  classrooms  is  that  in  learning  communities,  both  the  membeis 
themselves  and  the  collective  knowledge  and  skills  of  the  community  are  viewed  as  important  resources”  (p.4). 

One  significant  implication  of  the  organic  knowledge-building  model  is  that  individuals  will  bring  their 
challenges,  experiences,  and  ideas,  which  can  be  the  product  of  individual  problem-centered  learning  activities,  to  a 
discourse  and  action  community  and  then  extract  ideas  from  their  discussions  (and  vice  versa).  These  ideas  can  be 
used  back  in  their  workspaces  and/or  discussed  in  other  learning  communities.  This  is  a natural  process  that  people 
use  already  when  they  transfer  knowledge  to  and  from  workplace  settings,  school,  hobbies,  and  other  so-called 
compartments  of  their  lives.  In  fact,  compartmentalizing  one’s  knowledge  or  avoiding  dis  cussion  of  ideas  about 
which  one  is  passionate  is  an  unnatural  and  self-defeating  enterprise.  Knowledge  is  not  something  that  grows  by 
hoarding;  rather,  it  can  flourish  only  in  environments  that  enable  and  encourage  movement  of  ideas.  This  concept  is 
the  essence  of  organic  architecture  and  is  at  the  heart  of  the  organic  knowledge-building  model. 

Facilitation 

Given  its  dynamic  nature,  a learning  environment  without  a facilitator  is  little  more  than  a rudderless  ship 
adrift  on  an  ever-expanding  sea  of  information.  To  support  the  achievement  orientation  of  organic  knowledge- 
building, five  roles  associated  with  facilitators  have  been  identified  (Prestera  & Moller,  2001).  These  are  the 
facilitator  as  a guide,  mentor,  catalyst,  coach,  assessment-giver,  and  resource-provider.  Each  of  these  roles  is  a form 
of  feedback  that  the  facilitator  provides  in  the  context  of  an  organic  knowledge-building  environment. 

As  guides,  facilitators  help  individuals  form  strategies  for  navigating  the  rich,  and  sometimes  complex, 
learning  environment.  After  all,  individuals  who  are  new  to  asynchronous  learning  environments  are  on  multiple 
learning  curves  as  they  acclimate  to  new  technology,  communication,  socio -cultural,  and  learning  process  issues. 

As  mentors,  facilitators  model  the  values  of  the  learning  community.  As  catalysts,  facilitators  challenge  and 
stimulate  individuals  to  explore  and  interact. 
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As  coaches,  facilitators  provide  formative  feedback  and  encouragement.  As  assessment-givers,  facilitators 
create  opportunities  for  individuals  and  groups  to  showcase  their  newfound  skills  and  knowledge.  Facilitators  will 
also  evaluate  the  individual  less  in  terms  of  the  substance  of  the  individual’s  output  and  more  in  terms  of  the 
individual’s  learning  process.  Values  often  manifest  themselves  in  evaluation  rubrics.  These  rubrics  can  be  written 
to  focus  on  process  skills  as  well  as  content  skills.  For  example,  a rubric  might  include  an  evaluation  of  the 
reflection,  analysis,  and  collaboration  activities  in  which  the  individual  engaged.  If  one  goal  of  education  is  to 
produce  lifelong  knowledge-builders,  then  learners  must  demonstrate  the  ability  to  process  information  and  solve 
problems,  not  simply  the  ability  to  accumulate  knowledge. 

As  resource-providers,  facilitators  actively  support  learners  by  making  available  a wide  range  of 
information  objects,  scaffolds,  and  interaction  opportunities.  For  example,  it  sometimes  may  be  appropriate  to 
capture  the  experiences  of  expert  practitioners  in  a variety  of  situations  and  make  them  available  as  information 
objects  in  the  form  of  video  clips.  Or,  one  could  capture  the  reflections  of  one  student  who  struggled  through  a 
learning  process  and  make  that  available  as  a scaffold  to  help  other  learners  who  will  go  through  a similar  process. 
Similarly,  a facilitator  may  find  it  valuable  to  connect  learners  with  an  online  community  of  subject  matter  experts 
who  work  in  a contextually  relevant  field.  In  fact,  an  important  aspect  of  building  a community  should  be  building 
links  with  other  communities.  As  Brown  and  Duguid  point  out,  “Isolated  communities  can  get  stuck  in  ruts,  turning 
core  competencies  into  core  rigidities”  (p.  6). 

Facilitation  plays  a vital  role  within  the  organic  knowledge-building  model.  While  information  objects, 
scaffolds,  and  discourse  action  communities  give  substance  to  the  learning  environment,  facilitation  gives  it  shape 
and  form.  Without  facilitation,  a learning  community  is  merely  the  architectural  equivalent  of  a pile  of  bricks. 

Conclusion 

Technology  breakthroughs  have  always  held  the  potential  to  improve  life.  However,  it  is  only  as  we  truly 
seek  to  understand  the  vast  realm  of  possibilities  available  in  these  breakthroughs,  and  as  we  have  the  courage  and 
inventiveness  to  utilize  that  technology  in  creative,  heretofore  unexplored,  perhaps  controversial  but  always 
challenging  ways,  that  we  can  even  begin  to  realize  technology’s  potential  to  improve  the  quality  of  learning  and 
life.  Simply  recreating  the  present  in  a more  efficient  manner  seems  wasteful,  especially  when  the  status  quo  is  so 
ineffective.  By  itself,  technology  in  learning  is  a benign  force.  However,  when  tools  and  learning  strategies  are 
combined  and  constructed  in  ways  that  expand  our  potential  to  learn,  we  give  meaning  to  that  technology. 

Accessibility  to  technology  is  only  half  the  equation.  Technology  has  great  abilities  to  carry  and  present 
information;  however,  learning  is  more  than  receiving  and  storing  information.  Information  only  becomes 
knowledge  when  people  convert  it  through  application.  Human  beings  are  social  creatures  who  traditionally 
encourage,  communicate,  and  share  amongst  themselves. 
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Introduction 

Principled  design  experiments  are  necessary  to  delineate  a particular  domain  of  learning.  Principled  design 
experiments  are  those  experiments  which  take  place  over  multiple  cycles,  and  the  results  of  previous  cycles  serve  as 
controls  for  the  next  cycle  (McGee  & Howard,  1998).  This  method  reduces  the  need  for  control  groups.  Past  cycles 
now  serve  as  the  “control”  with  which  future  measurements  can  be  compared  (Bunderson,  2002).  Knowing  each 
learner’s  current  status  of  learning  (as  a result  of  the  continuous  evaluation  within  each  cycle)  reduces  the  need  for 
randomization.  Randomization  assumes  equality,  whereas  principled  design  experiments  know  exactly  where  the 
learners  are,  without  having  to  guess  their  current  status.  A domain  of  learning  is  a category  such  as  math,  science, 
literature,  foreign  language,  manufacturing,  etc.  A delineated  domain  is  one  where  we  can  tell  within  the  domain 
what  is  easy,  what  is  somewhere  between  easy  and  hard,  and  what  is  hard  (Bunderson,  2002).  Once  a domain  is 
delineated,  we  can  begin  to  measure  progress,  give  meaningful  feedback,  and  instruct  how  to  rapidly  progress  from 
lower  levels  to  higher  levels  in  the  given  category  (domain).  This  is  how  principled  design  experiments  increase 
performance  within  a domain  of  learning.  By  using  principled  design  experiments  we  delineate  a domain,  and  then 
efficiently  increase  performance  within  that  domain  by  providing  meaningful  instruction,  evaluation,  and  feedback. 
The  instruction,  evaluations,  and  feedback  are  all  linked  (connected)  to  the  particular  construct.  A construct  is  a 
section,  milestone,  or  benchmark  within  the  domain  (category  of  learning),  and  it  can  be  either  easy,  somewhere 
between  easy  and  hard,  or  hard  to  perform  (Bunderson,  2002).  These  milestones  are  what  are  clearly  delineated  in  a 
principled  design  experiment.  In  a principled  design  experiment,  it  becomes  apparent  that  certain  prerequisite 
constructs  (sections,  milestones,  or  benchmarks)  must  be  mastered  in  order  to  progress  to  the  next  construct.  Thus, 
in  order  to  provide  meaningful  instruction,  evaluation,  and  feedback,  and  ultimately  to  rapidly  increase  performance, 
it  becomes  necessary  to  develop  “construct-linked”  scales  of  measurement.  These  scales  of  measurement  are 
connected  to  the  clearly  delineated  milestones  or  benchmarks  (constructs).  Instruction  and  feedback  that  is  linked  or 
connected  to  all  the  skills  in  these  construct -linked  scales  is  meaningful  and  useful,  and  makes  for  efficient  progress 
by  eliminating  unnecessary  activities  that  are  not  tied  to  the  construct  linked  scales  and  ultimately  have  no  effect  on 
an  individual’s  progress  within  the  domain. 

In  order  for  construct  linked  (benchmark -connected)  scales  to  be  useful,  they  must  not  vary  or  change  in 
certain  ways.  In  other  words,  they  must  be  invariant  in  certain  ways.  By  being  linked  to  a construct,  a scale  already 
possesses  one  type  of  invariance  - construct  invariance.  It  is  tied  to  an  unchanging  benchmark,  instead  of  a 
statistical  norm.  Other  types  of  invariance  that  are  important  are  discussed  in  this  paper.  Principled  design 
experiments,  delineated  domains,  and  construct -linked  invariant  scales  of  measurement  are  a powerful  method  to 
increase  performance  and  progress  in  a rapid  and  efficient  manner.  This  paper  explains  the  elements  that  make  up 
this  powerful  method  by  describing  them  in  detail.  Each  element  of  this  method  is  an  essential  piece,  which,  like  a 
house  or  a puzzle,  is  incomplete  if  it  is  lacking. 

Preliminary  Steps  to  the  Principled  Design  Experiment 

Before  beginning  a principled  design  experiment,  some  important  steps  need  to  be  taken.'  A skilled 
teacher  or  trainer  will  readily  identify  these  skills  as  those  which  are  necessary  to  develop  an  effective  course. 
Differences  in  these  steps  from  other  approaches  are  mainly  in  the  depth  of  detail  necessary  for  a principled  design 
experiment  and  in  the  special  attention  that  is  paid  to  the  ordering  of  tasks.  The  initial  steps,  therefore,  involve 
collecting  and  ordering  the  information  in  a particular  course,  discipline,  or  sub-discipline  (domain).  Subject  matter 
experts  are  particularly  useful  at  this  step,  to  aid  in  defining  and  prioritizing  the  information.  Specifically,  they  can 
help  to  determine  the  following: 

• Divide  the  topics  into  subheadings.  This  is  the  first  step  in  completely  understanding  the  category  of  learning 
(domain),  and  how  to  determine  the  levels  of  expertise  of  the  students. 
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• Sequence  the  subheadings.  This  is  necessary  in  order  to  know  how  to  effectively  instruct  and  present 
information  logically  and  hierarchically,  and  to  ensure  that  prerequisite  skills  are  mastered  and  understood  in 
order  for  future  learning  to  be  meaningful  and  effective.  This  step  helps  to  determine  which  subheadings  are 
easy,  which  are  somewhere  between  easy  and  hard,  and  which  are  hard.  It  also  becomes  clear  which 
subheadings  can  be  taught  before  or  after  other  subheadings.  For  example,  in  order  to  understand  Calculus,  it  is 
necessary  to  understand  algebra.  The  subheading  (topic)  of  geometry,  however,  may  be  taught  either  before  or 
after  algebra. 

• Initially  place  subheadings  into  categories.  Where  categories  are  defined  as  a set  of  subheadings  that  logically 
fit  well  together  into  a course,  they  need  to  be  placed  and  ordered.  For  example,  consider  the  course  English  as 
a second  language  (ESL).  Subheadings  should  be  divided  up  into  the  four  categories  of  reading,  listening, 
speaking,  and  writing. 

• Develop  an  initial  understanding  of  what  constitutes  various  levels  of  expertise.  Since  the  goal  is  to  develop  a 
robust  scale  of  learning  progress  that  can  be  re-used  over  and  over,  there  must  be  a good  initial  understanding  of 
the  effort  required  to  progress  from  what  is  easy,  to  what  is  somewhere  between  easy  and  hard,  to  what  is  hard. 

In  essence,  the  pre -principled  design  experiment  work  enables  the  instructor  to  identify  the  content  of  her 
course,  provide  a rationale  for  content  inclusion,  and  outline  a way  to  measure  and  improve  student  progress.  Ear  too 
frequently  in  education  we  see  a “shotgun”  approach,  where  an  instructor  has  a somewhat  good  idea  of  which 
categories  should  be  included  in  the  topic,  but  lacks  the  properly  identified  content  and  its  rationale.  In  this  situation, 
the  particular  instructor  fails  to  recognize  the  importance  of  linking  and  ordering  the  skill  set  and  knowledge  base 
required  to  evaluate  student  learning.  The  pre -principled  design  experiment  work  is  the  first  step  in  connecting 
categories  to  level  of  difficulty,  and  then  level  of  difficulty  to  the  student’s  achieved  level  of  expertise. 

Bethanie  Newby’s  (2002)  work-in-progress  is  an  example  of  the  preparation  steps  or  the  pre -principled 
design  experiment  work.  Her  topic  is  an  introductory  course  on  forced  migration.  This  course  entails  people  living  in 
Third  World  countries  who  are  forced  to  migrate  due  to  famine,  political  unrest,  social  conflict,  and/or  war.  After 
selecting  this  course,  she  created  a list  of  content  she  felt  would  be  important  to  include  (i.e.,  the  subheadings  of  the 
topic).  With  the  use  of  the  internet,  she  was  then  able  to  obtain  additional  assistance  by  observing  the  topics  others 
teach  in  similar  courses.  Table  1 below  provides  a synopsis  of  how  the  topics  were  chosen. 


Question  asked  of  experts: 

In  an  introductory  university-level  course  on  forced  migration,  what  topics  do  you  feel  are  important  to 
cover?  Choose  all  that  apply. 


Movement  of  people  across  national  boundaries 

Illegal  migration 

Trafficking 

Who  are  the  migrants 

Conceptualizing  migration 

Consequences  of  migration  on  sending  and  receiving  states 

Duties  and  responsibilities  of  states 

Policies  of  controlling  entry 

Policies  concerning  citizenship  and  nationality 

Definition  of  “forced  migrants”  (refugees  and  the  internally 

displaced) 

Responses  of  asylum  countries 

Role  of  international  and  non-governmental  organizations  in 
protecting  and  assisting  forced  migrants 
Consequences  of  forced  migration  on  the  environment 
Consequences  of  forced  migration  on  national  security 
Coping  mechanisms,  survival  strategies,  and  the 
psychological  adaptation  of  affected  populations 
Vulnerable  groups  within  forced  migrant  population 
Other  ? 


Labor  migration 
Forced  migration 

Migration  changes  since  the  end  of  the  Cold  War 

Why  do  people  migrate 

Global  networks  arising  from  migration 

Impact  of  migration  on  international  institutions 

Duties  and  responsibilities  of  international  organizations 

Absorption  and  integration  of  migrants 

Asylum  policies 

“Root  causes”  of  displacement 

Role  of  governments  in  protecting  and  assisting  forced  migrants 
Psychosocial  needs  of  forced  migrants 

Consequences  of  forced  migration  on  the  public  health 

Impact  of  forced  migrants  on  host  populations  and  governments 

Resettlement 

Physical  (basic)  needs  of  forced  migrants  - food,  safety,  shelter 


Table  1.  Newby,  B.  (2002).  Topics  for  an  introductory  Forced  Migration  Course 
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Newby’s  (2002)  next  step  was  to  have  the  experts  rank  the  knowledge  and  skills  required  to  understand 
forced  migration.  Table  2.  This  involved  dividing  the  various  ability  levels  by  their  respective  ranking;  see  Table  3 


Level 

Milestone 

Beginning 

Low  Intermediate 

Intermediate 

Can  explain  concepts  and  terms.  Understands  scope  (numbers,  locations  affected) 

Understands  causes  of  forced  migration. 

Can  explain  moral  and  ethical  dilemmas.  Can  analyze  and  rank  causes.  Can  analyze  situat  ions  and  explain 

Advanced 

their  significance.  Can  participate  effectively  in  a non-leadership,  professional  position. 

Can  assess  and  design  strategies,  programs,  and  policies.  Can  make  predictions  of  probable  outcomes.  Can 
monitor  and  evaluate  situations. 

Table  2.  Newby,  B.  (2002).  Initial  ranking  of  forced  migration  constructs 


B=Beginning 

L = Low  Intermediate  I = Intermediate  A = Advanced 

Example:  B 

What  is  a refugee? 

Rankins 

Task 

Estimate  the  current,  total  number  of  forced  migrants 
Identify  target  populations,  or  sub-groups,  that  are  likely  to 
migrate 

Describe  the  characteristics  of  a forced  migrant 

Explain  the  differences  between  refugees  and  internally  displaced 

persons 

Explain  asylum,  resettlement,  and  repatriation 
Discuss  the  “root  causes”  of  forced  migration 
What  additional  factors  cause  people  to  migrate? 

Describe  national  and  international  responses  to  forced  migration 
For  a given  situation,  analyze  the  causes  of  migration  and 
identify  the  most  vulnerable  groups 
Identify  the  major  players  involved  and  discuss  how  their 
involvement  affects  the  situation 

Volunteer  at  a local  refugee  centre  to  help  process  clients  and 
direct  them  to  appropriate  services 

Volunteer  at  a local  refugee  centre  to  write  press  releases  and 
news  articles 

Analyze  asylum  policy  for  Country  A,  identify  the  five  most 
common  groups  or  types  of  migrants  who  seek  asylum  there,  and 
explain  the  probable  outcomes  of  their  asylum  requests 
Design  population,  health,  environmental,  and/or  psychosocial 
programs  or  projects  in  the  context  of  national  and  international 
policy 

Incorporate  cultural  factors  in  the  design  and  management  of 
proj  ects/programs 

Evaluate  development  programs/projects  in  terms  of  their 

quality,  effectiveness,  and  efficiency  using  qualitative  and 

quantitative  research  techniques 

Research,  design  and  promote  asylum  policy 

Assess  critically  strategic  obstacles  to  successful  repatriation 

programs/projects 

Use  state-of-art  information  technologies  in  the  design  and 
monitoring  of  programs/projects 

Access  and  utilize  effectively  the  programming  resources  of 
local,  national,  and  international  organizations  engaged  in  forced 
migration  activit  ies 

What  situations  do  traffickers  exploit  to  bring  themselves  to 
profit 

Table  3.  Newby,  B.  (2002).  Assignment  of  skill  levels  to  tasks 

Newby’s  final  step  in  preparation  for  an  actual  principled  design  experiment  with  students  - Table  4 - was 
to  have  the  experts  list  the  tasks  from  the  previous  step  in  a two-dimensional  table,  where  columns  are  representative 
of  proficiency  of  the  person(s),  and  rows  indicate  the  level  of  difficulty  of  the  tasks. 
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Less  Proficient 

More  Proficient 

Most  Proficient 

Easy 

Medium 

Hard 

Table  4.  Newby,  B.  (2002).  Ranking  tasks  by  student  proficiency  and  task  difficulty 


Upon  completion  of  these  steps,  Newby  is  in  an  ideal  position  to  begin  a design  experiment.  Expertise  can 
now  be  measured  in  an  introductory  course  on  forced  migration. 

This  actual  example  illustrates  the  pre -principled  design  work.  The  efficacy  of  this  course  is  borne  out  in 
the  actual  experiment.  A live  classroom  setting  will  finally  determine  what  needs  to  be  changed  or  modified.  Thus, 
there  is  an  interaction  between  theory  and  data. 

The  Principled  Design  Experiment 

Upon  completion  of  defining  the  content  of  the  course  and  its  constituent  subheadings,  ordering  the 
subheadings  in  terms  of  difficulty,  and  defining  what  constitutes  the  respective  levels  of  knowledge,  the  instructor 
can  begin  the  principled  design  experiment.  This  experiment’s  primary  purpose  is  to  delineate  the  domain 
rigorously,  or  map  out  the  process  of  learning  the  course  and  create  a learning  progress  map.  As  proposed  by  the 
authors,  the  design  experiment  involves  unobtrusive  methods  for  obtaining  measures  of  progress.  These  unobtrusive 
measures  come  from  practice  sessions  with  feedback.  Experimentation  and  research  takes  place  in  live  settings  in 
which  repetitive  cycles  of  implementation  and  carefully  principled  design  experiments  are  used. 

A few  iterations  of  the  design  experiment  may  be  required  before  the  learning  progress  map  is  established. 
We  avoid  saying  that  the  map  is  finished  or  completed  because  it  will  continue  to  change  and  evolve  over  time. 

Thus,  our  learning  progress  map  requires  continual  and  ongoing  examination  to  ensure  that  it  is  still  valid.  Later 
sections  in  this  paper  detail  this  method  of  ongoing  examination. 

Once  categories  of  the  course  are  identified,  testlets  are  used  to  check  levels  of  understanding  and 
knowledge  gained  in  each  milestone.  Testlets  are  assessments  of  the  individual  benchmarks  that  comprise  the 
domain.  These  mini-tests  are  directed  at  one  milestone  (construct)  and  are  linked  to  each  other.  The  instructor  can 
readily  determine,  through  the  results  of  each  testlet,  how  well  each  student  comprehends  a particular  construct. 
Testlets  are  created  with  the  following  characteristics: 

• Test  students  knowledge  and  understanding  in  only  one  category 

• Simple  to  create,  as  there  is  a need  for  many  of  them  in  order  to  have  continuous  evaluation  (an  essential  part  of 

developing  a domain  theory  of  learning) 

• Easily  administered 

• Unobtrusive 

• Given  many  times  during  the  duration  of  the  course 

The  results  of  previous  principled  design  experiment  iterations,  as  obtained  by  the  testlets,  serve  as  controls 
for  the  next  cycle.  This  reduces  the  need  for  control  groups.  This  is  possible  since  the  process  began  with  a solid 
initial  model  developed  by  subject  matter  experts.  We  use  the  design  experiment  to  both  observe  how  students 
compare  with  the  model,  and  to  see  how  the  model  holds  up  to  the  results  and  comments  from  the  students.  Thus, 
there  is  an  interaction  between  theory  and  data.  These  results  help  the  instructor  perform  necessary  modifications 
of  the  model  before  beginning  another  iteration  of  the  principled  design  experiment. 

In  the  pre -principled  design  experiment  phase,  the  instructor  may  have  contrived,  for  example,  a link 
between  a category  and  an  ability  level.  In  the  case  of  the  course  on  forced  migration,  the  consensus  of  the  subject 
matter  experts  was  that  a novice  student  should  be  able  to  define  forced  migration,  identify  the  population,  list  and 
describe  distinctions,  and  understand  the  scope  of  the  problem.  This  category  could  rightly  be  called  “What  is 
forced  migration?”  The  instructor  would  then  design  a curriculum  for  this  category,  making  sure  to  include  all 
relevant  topics,  and  any  needed  hands-on  practice.  At  this  point,  the  instructor  would  design  the  testlets  for  each 
topic  in  the  category,  and  add  more  complexity  as  needed  to  capture  the  progression  in  learning.  Testlet  results 
will  indicate  the  efficacy  of  the  curriculum  in  producing  the  desired  level  of  mastery,  and  will  also  pinpoint  any 


314 


needed  adjustments.  They  will  also  show  if  the  pace  is  appropriate.  While  design  experiments  are  primarily  geared 
toward  the  measurement  of  learner  progress,  they  may  also  be  used  to  strengthen  curriculum  development.  In 
order  to  create  an  effective  design  experiment,  the  instructor  must  gather  information  during  the  pre -design  phase 
that  will  address  the  following  questions: 

• What  knowledge  should  be  gained  in  each  category? 

• What  cognitive  links  should  the  student  be  able  to  make  with  the  knowledge  gained? 

• What  behavior,  performance,  or  functional  skill  should  the  student  be  able  to  apply  upon  mastering  this 

category? 

Once  the  curriculum  has  been  developed  using  the  dual  model  of  understanding  and  performance  (a  central 
aspect  of  domain  theory),  the  design  experiment  can  be  initiated.  At  this  point,  the  instructor  must  then  create  testlets 
that  will  accurately  measure  the  knowledge  and  ability  each  student  gains  in  the  given  milestone.  This  is  the  first 
step  in  creating  a valid  measurement  scale.  The  measurement  scale  is  necessary  for  determining  valid  results  and  for 
future  replication(s). 

The  resultant  delineated  domain  of  learning  indicates  which  milestones  are  easy,  which  ones  are  between 
easy  and  hard,  and  which  ones  are  hard  (Bunderson,  2002).  The  domain  theory  (learning  progress  map  with 
delineated  benchmarks  or  milestones)  contains  a series  of  categories  (sets  of  subheadings).  Each  principled  design 
experiment  should  help  us  perfect  the  contents  of  each  category.  Thus  there  is  repeated  interaction  between  theory 
and  data,  as  they  continually  refine  one  another.  A principled  design  experiment  helps  us  answer  the  following 
questions  regarding  subheadings: 

• Do  we  have  the  proper  subheadings  in  each  category? 

• Do  these  seem  to  fit  together? 

• Should  they  be  divided  in  a slightly  different  way? 

In  addition  to  understanding  the  placement  of  subheadings,  and  their  logical  ordering  scheme,  each 
principled  design  experiment  also  helps  us  understand  the  amount  of  effort  required  to  progress  from  one  milestone 
to  the  next. 

The  Need  for  Proper  Measurement  and  Invariant  Scales 

Throughout  the  principled  design  experiment,  proper  measurement  is  essential  to  improve  student  progress. 
Improper  measurement  can  prevent  progress  in  many  different  ways.  False  progress  reports  may  erroneously  lead  an 
instructor  to  believe  that  certain  teaching  methods  are  effective  at  increasing  performance,  and  may  give  students 
confusing  and  misleading  results.  Properly  measuring  the  impact  of  teaching  cannot  be  overemphasized. 

Consider  an  example  concerning  a method  of  assigning  numbers  from  1 to  7 to  people  based  on  their 
abilities  to  swim  (see  also  an  excellent  example  in  Fisher,  1993).  The  higher  the  number,  the  greater  the  ability.  For 
example,  the  numbering  system  might  go  from  (1)  no  fear  of  the  water  to  (2)  dog  paddle  to  (3)  floating  to  (4) 
somehow  making  it  across  a pool  to  (5)  correctly  performing  the  kick  for  the  stroke  called  freestyle  to  (6)  correctly 
performing  the  arm  movements  for  freestyle  to  (7)  correctly  doing  at  the  same  time  the  leg  and  arm  movements  for 
freestyle.  Julie  might  get  a 7 because  she  can  swim  the  freestyle  stroke  in  perfect  form.  Diane  might  receive  a 5 
because  she  can  do  the  leg  action  of  freestyle,  but  not  the  arm  action.  Cheryl  might  receive  a 3 because  she  can  float 
on  her  back.  Tom  might  receive  a 1 because  he  is  not  afraid  of  the  water,  even  though  he  would  immediately  drown 
if  he  fell  in  over  his  head.  When  the  rating  scale  is  used,  the  numbers  are  frequently  recorded  with  equal  distance 
between  them  as  such: 

1 2 3 4 5 6 7 

This  scale  gives  the  impression  that  the  numbers  represent  equal  increases  in  ability.  Is  it  necessarily  true 
that  adding  two  units  of  ability  to  Tom  who  was  given  the  rating  of  1 will  get  him  up  to  the  level  of  Cheryl  who  was 
given  the  rating  of  3?  Will  adding  two  units  of  ability  to  Diane  who  was  rated  at  5 move  her  to  the  ability  level  of 
Julie,  who  was  rated  at  a 7?  No!  They  are  numbers  used  improperly.  This  is  an  example  of  improper  measurement, 
and  of  an  error  that  occurs  frequently.  While  a discussion  of  the  fundamental  theory  of  representation  is  beyond  the 
scope  of  this  paper,  it  is  sufficient  to  say  that  this  method  of  measurement  violates  it.  Numbers  are  erroneously  used 
to  represent  qualitative  things. 

Taking  this  example  further,  it  may  be  possible  that,  when  analyzed  as  the  difficulty  of  moving  from  one 
category  to  the  next,  the  numbers  could  be  spaced  as  follows: 

1 234  5 6 7 

Adding  the  numbers  is  now  meaningless.  Adding  them  doesn’t  tell  you  anything.  Perhaps  you  may  feel 
embarrassed  that  you  have  done  something  similar.  Take  comfort  in  the  fact  that  you  are  not  alone.  People  in  nearly 
all  professions  do  it  all  the  time.  People  who  receive  test  scores  twice  as  high  as  other  people  are  commonly  falsely 
assumed  to  be  “twice  as  smart”,  and  so  forth.  However,  once  it  is  known  that  the  numbers  in  our  example  are  not 
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equally  spaced,  it  not  only  makes  no  sense  to  add  them,  it  also  makes  no  sense  to  apply  any  sort  of  mathematics  to 
them.  Nor  is  it  possible  to  perform  any  statistical  analysis,  hypothesis  testing,  ANOVA,  etc.  With  improper 
measurement  that  is  not  calibrated  to  properly  reflect  the  distance  between  the  delineated  constructs  in  a domain,  we 
are  left  with  numbers  that  are  useless  in  measuring  progress  and  improving  performance. 

How  can  this  problem  of  measurement  be  solved?  How  can  progress  be  measured?  The  following  steps 
below  explain  a practical  method  for  helping  to  remedy  this  problem. 

1.  Replace  the  numbers  above  with  letters,  so  there  is  no  temptation  to  do  anything  with  them  such  as 
addition. 

2.  Develop  a scale  for  effort.  This  will  require  time,  as  many  observations  of  various  levels  of  effort  must 
be  made  to  calibrate  the  scale. 

3.  Put  the  category  letters  on  top  of  the  newly  created  and  calibrated  scale. 

You  can  now  much  more  accurately  measure  the  required  amount  of  effort  needed  to  get  from  one  category  to 
another.  The  end  result  might  look  something  like  this  below: 


We  can  now  attempt  to  measure  progress  with  this  re -calibrated  scale.  A person  at  category  level  D needs  to  exert  4 
units  of  ability  to  move  to  category  E,  while  a person  at  level  F needs  to  exert  only  2 units  of  ability  to  progress  to 
category  G. 

Proper  measurement  opens  many  doors  to  improvements  in  instruction  and  learning.  Once  measurement  is 
occurring  properly,  “ . . .we  will  be  able  to  describe  abilities,  measure  growth,  discover  interrelationships,  examine 
rates  of  learning,  diagnose  difficulties,  conduct  experiments  without  the  expense  of  simultaneous  control  groups  and 
more  effectively  share  and  compare  our  experimental  findings  with  others  examining  learning  in  the  same  domain 
(Pelton,  2001).” 

As  can  be  seen  through  the  example  above,  proper  measurement  involves  the  development  of  a scale.  The 
more  accurate  the  scale,  the  better  the  results.  In  most  cases  in  education,  any  improved  scale  will  be  better  than 
most  current  measurement  practices.  Before  we  could  measure  progress  in  the  example  above,  we  needed  a 
calibrated  scale  of  physical  ability.  The  development  of  scales  for  learner  progress  will  be  discussed  in  detail  in  the 
next  section. 

Developing  a Measurement  Scale  for  Learner  Progress 

The  development  of  a measurement  scale  is  a time  intensive  process.  Nevertheless,  its  development  is  very 
rewarding  in  aiding  both  the  instructor  and  the  student  to  track  and  improve  learner  progress.  It  should  also  be  fairly 
easy  to  modify  over  time  and  across  situations,  as  circumstances  and  attitudes  change.  The  scale  is  different  for 
every  topic  or  course  of  learning,  as  no  one  scale  fits  all.  The  example  used  in  this  section  concerns  English  as  a 
second  language  (ESL)  speaking  ability. 

Strong-Krause  (2001)  developed  a scale  for  learner  progress  in  ESL  communicative  competence.  Her  work 
resulted  in  a process  for  scale  development,  a process  which  can  be  replicated.  This  process  is  called  construct- 
linked  scale  development  (CLSD).  The  four  steps  of  CLSD  are  shown  below  in  the  ESL  speaking  ability  example. 

The  first  step  is  construct  delineation,  which  breaks  down  the  topic  or  course  into  various  categories 
(constructs,  subheadings,  or  benchmarks).  It  is  in  this  stage  where  connections  between  observed  phenomena  are 
beginning  to  emerge,  and  the  instructor  at  this  point  develops  a possible  reason  for  these  connections.  Strong-Krause 
broke  down  the  ESL  speaking  ability  course  into  ten  task  types  with  four  variations  within  each  task  type.  For 
example,  one  of  the  ten  task  types  was  narrating  a personal  story.  The  four  variations  of  this  task  type  were  to  tell 
about  a recent  vacation,  tell  about  a difficult  situation,  tell  about  an  experience  as  a child,  and  tell  about  a movie. 
Another  task  type  was  to  give  an  opinion.  Once  these  tasks  or  constructs  were  delineated,  they  were  then  able  to  be 
ordered. 

Construct  linked  ordering  - the  second  step  of  CLSD  - starts  when  levels  of  expertise  begin  to  be 
understood  qualitatively  in  their  logical  order.  Logical  ordering  of  tasks  consists  of  sorting  physical  or  mental 
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performances  according  to  their  levels  of  difficulty.  Strong-Krause  (2001)  initially  ordered  her  ten  task  types  as 
follows,  from  easiest  to  most  difficult: 

1.  Naming  common  objects 

2.  Giving  personal  information 

3.  Giving  information  about  others 

4.  Dealing  with  typical  social  situations 

5.  Asking  questions 

6.  Narrating  a personal  story 

7.  Narrating  a story  given  a visual  prompt 

8.  Dealing  with  a complication  in  a social  setting 

9.  Telling  about  the  future 

10.  Supporting  an  opinion 

In  order  for  the  delineation  and  ordering  to  have  an  effect  beyond  one  group  of  students,  the  third  step  must  be 
carried  out,  which  is  developing  a scale  which  is  unchanging  in  certain  ways:  an  invariant  scale. 

Although  human  emotions,  intentions,  desires  and  needs  change  from  day  to  day,  we  can,  with  a high  level 
of  reliability,  develop  a measurement  scale  that  won’t  vary  much  over  time.  We  call  this  unchanging  scale  an 
invariant  scale.  Invariant  scale  development  is  the  third  step  in  CLSD.  It  is  an  essential  step  that  must  be  developed. 
Without  an  invariant  scale,  our  scale  would  be  flimsy  at  best.  Think  of  the  example  given  earlier  about  swimming 
ability.  Over  time  experts  in  teaching  swimming  could  develop  a relatively  unchanging  unit  of  effort  that  they  could 
use  to  scale  ability  to  swim.  It  could  give  them  the  measuring  rod  they  need  to  measure  and  improve  progress.  If  it 
were  not  relatively  unchanging  over  time,  it  would  be  much  more  difficult  for  them  to  measure  and  predict  progress 
in  swimming.  They  could  not  accurately  compare  teaching  methods  nor  rates  of  improvement.  It  would  be  more 
difficult  to  isolate  cause  and  effect,  diagnose  potential  complications,  and  discover  interrelationships  among 
teaching  methods.  A robust  measuring  rod  helps  in  many  ways. 

When  an  invariant  scale  is  developed,  comparable  measurements  can  be  taken,  regardless  of  the  group  of 
students  taking  the  course.  This  is  called  sample  invariance.  Furthermore,  testlets  used  to  gauge  skill  level  can 
change  without  affecting  the  scale,  as  long  as  the  testlets  continue  to  cover  the  same  category  in  a similar  fashion. 
This  is  called  question  invariance.  A third  type  of  invariance  is  that  the  scale  has  equal  intervals.  Invariance  can  be 
assured  through  the  iterations  of  the  principled  design  experiment.  Each  iteration  will  continue  to  help  fine  tune  the 
scale. 

Strong-Krause  (2001)  used  experts  in  the  field  of  ESL  to  develop  a theoretical  scale.  The  scale  she 
developed  is  shown  below  in  Eigure  1 . The  numbers  in  the  boxes  coincide  with  those  of  the  ten  task  types  listed 
above  in  their  order  of  difficulty. 
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FIGURE  1.  The  theoretical  distribution  of  the  ten  ESL  task  types  on  an  equal-interval  scale  from  1 to  10. 

The  fourth  step  of  CLSD  - called  construct-linked  scaling  - is  where  task  difficulty  is  linked  to  level  of 
expertise.  “The  development  of  a measurement  scale  provides  the  foundation  to  predict  which  skills  are  needed  to 
complete  tasks  at  a particular  level  of  expertise’’  (Strong-Krause  (2001),  p 28).  In  the  case  of  the  ESL  example,  it  is 
the  relationship  between  the  underlying  language  abilities  and  the  performance  on  tasks,  which  determines  the 
degree  of  mastery  of  those  underlying  language  abilities  that  can  be  inferred.  To  continue  using  the  example  of  ESL, 
Strong-Krause  used  the  tasks  identified  above  to  help  forma  set  of  seven  skill  combinations.  These  skill 
combinations  were  statistically  matched  to  a scale  that  rates  proficiency.  The  seven  skill  combinations  are  listed  as 
follows: 
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Skill  A:  Use  simple  structures  with  sentence-level  discourse  and  most  basic  vocabulary. 

Skill  B:  Use  simple  structures  with  sentence-level  discourse  with  strong  basic  vocabulary. 

Skill  C:  Use  relatively  complex  structures  with  familiar  tasks. 

Skill  D:  Use  simple  structures  with  paragraph-level  discourse. 

Skill  E:  Use  relatively  comple  x structures  on  a less  familiar  task  with  a solid  vocabulary. 

Skill  F:  Use  relatively  complex  structures  on  a less  familiar  task  with  solid  basic  and  some  academic  vocabulary. 
Skill  G:  Use  relatively  complex  structures  on  a less  familiar  task  with  a strong  basic  vocabulary,  academic 
vocabulary,  and  specific  content  words. 

Strong-Krause  statistically  linked  the  seven  skill  combinations  to  four  different  levels  of  examinee 
proficiency.  This  is  shown  in  Table  5. 
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Table  5.  Expected  ratings  for  skill  categories  by  examinee  proficiency  level. 

The  numbers  in  the  table  represent  ratings,  and  have  the  following  general  interpretation:  A rating  close  to  20 
indicates  that  an  examinee  at  the  specified  proficiency  level  cannot  complete  tasks  requiring  the  described  language 
skills.  A rating  close  to  40  indicates  the  examinee  is  developing  the  required  skills.  A rating  close  to  60  indicates  the 
examinee  can  perform  the  tasks  that  require  the  specified  skills  but  with  distracting  errors.  A rating  close  to  80 
indicates  the  examinee  can  perform  the  tasks  that  require  the  specified  skills  with  minor  language  errors.  As  can  be 
seen,  the  table  flows  as  expected.  For  example,  an  intermediate  level  person  finds  it  increasingly  more  difficult  to 
perform  the  ever  increasingly  more  difficult  skills,  from  Skill  A through  Skill  G. 

Thus,  the  end  result  of  the  construct-linked  scale  development  is  an  ability  to  link  task  difficulty  to  the 
ability  of  the  students,  and  to  do  so  in  a measurable  quantifiable  way.  This  takes  time  to  perform,  and  usually  a few 
iterations  before  modifications  from  one  iteration  to  the  next  become  minor.  Over  time,  the  process  must  be  repeated 
periodically,  as  various  factors  such  as  traditions  and  levels  of  importance  and  emphasis  change,  thus  affecting  the 
validity  of  the  scale  - the  topic  of  the  next  section. 

Validity- centered  Design 

The  model  of  learner  progress  will  go  through  many  interpretations  and  uses.  As  a consequence,  it  is  never 
validated  once  and  for  all.  The  rationale  for  the  model,  the  construct-linked  scales,  and  the  resultant  findings  and 
parameters  and  their  uses  must  be  validated  over  and  over  again.  Validation  must  be  at  the  center  of  the  design  of 
the  learner  progress  model.  “Validity -centered  design  as  we  currently  understand  it  is  the  beginning  of  a principled 
[i.e.,  rigorous]  design  process  for  designing  and  developing  improved  [models],  the  construct-linked  scales 
associated  with  them,  and  documenting  the  evidence  for  a validity  argument’’  (Bunderson,  2002,  p 12;  see  also 
Martinez,  Bunderson,  and  Wiley,  2000).  Validity-centered  design  consists  of  three  major  categories. 

Category  I is  the  design  for  usability,  appeal,  and  positive  expectations.  If  the  overall  learner  progress 
model  doesn’t  look  good  and  feel  good  to  the  instructor  and  the  students,  it  might  become  suspect.  They  might 
question  whether  or  not  it  actually  does  what  it’s  supposed  to  do.  Suspicion  about  a learner  progress  model  will 
begin  to  have  a negative  impact  on  the  model  itself.  Nonetheless,  this  category  is  not  verified  in  a rigorous  method 
of  statistical  analysis,  it  is  a more  qualitative  examination. 

Category  II  is  the  design  for  inherent  construct  validity.  Construct  validity  is  the  link  between  reality  and 
the  scores  or  measures  produced  by  the  instructor.  If  this  link  between  reality  and  scores  has  been  modified,  then 
tests  for  construct  validity  should  pick  it  up.  Suppose  that  through  the  process  of  time  changes  in  society  or  human 
nature  or  whatever  cause  a shift  in  the  purpose  for  the  course  or  the  points  that  the  course  should  emphasize.  Then,  it 
won’t  be  valid  anymore  for  its  original  purpose,  and  will  need  to  be  recalibrated  before  it  can  become  valid  again. 
These  recalibrations  reflect  changes  in  the  milestones  or  constructs  in  the  domain. 

Category  III  is  the  design  for  reliability  and  evidence  of  criterion-related  validity.  We  try  to  answer 
questions  such  as  the  following:  Can  our  results  be  generalized  to  a different  gender  mix,  racial  group,  local  group. 
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etc.?  Are  there  external  evidences  for  the  established  learner  progress  model?  Do  we  in  fact  see  over  time  the  results 
we  anticipated?  Once  validity  is  established  to  a certain  degree,  and  we  are  measuring  progress  over  time  more 
properly,  we  will  likely  encounter  yet  another  phenomena  while  progressing  in  the  domain  of  learning.  Performance 
may  actually  dip  before  it  gets  better.  This  is  a normal  occurrence  known  as  the  “J”  curve  of  implementation. 

The  /-curve  of  Implementation 

An  interesting  consequence  of  implementing  a new  model  into  an  existing  course  is  that  we  frequently 
experience  a dip  in  the  outcome.  This  is  a natural  thing  to  expect,  so  we  shouldn’t  be  surprised.  There  are  many 
reasons  for  this,  but  most  can  be  summarized  under  the  need  for  the  team  of  teachers,  assistants,  students,  and 
administrators  to  learn  the  new  system  and  to  make  the  new  process  gel.  We  call  it  the  7-curve  effect  because  - like 
the  shape  of  a capital  cursive  7-  outcome  might  go  down  first  before  it  starts  to  pick  up.  See  Figure  2. 
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FIGURE  2.  The  J -curve  of  Implementation 

An  example  of  this  occurred  to  Vic  Bunderson  several  years  ago  while  implementing  a new  process  at 
Brigham  Young  University  (BYU)  called  Time-Shared,  Interactive,  Computer-Controlled  Information  Television 
(TICCIT).  The  English  Grammar  and  Composition  course  that  was  on  TICCIT  was  used  in  several  sections  of 
English  to  prepare  students  to  take  the  university-wide  general  education  writing  exam.  Surprisingly,  there  was  only 
one  standard  measurement  system  at  BYU  for  all  sections  in  the  particular  domain  of  English  Grammar  and 
Composition.  The  baseline  average  was  55%  across  the  university.  After  the  first  semester,  only  45%  of  those  who 
followed  the  instructor  using  the  TICCIT  approach  passed  the  General  Education  (GE)  exam. 

The  team  of  developers  and  researchers,  working  with  the  English  instructor  who  was  using  TICCIT, 
developed  better  implementation  plans  for  the  second  semester.  At  the  end  of  the  second  semester  72%  of  the  class 
following  the  TICCIT  method  passed.  Eurther  implementation  changes  made  between  the  second  and  third 
semesters  lead  to  84%  of  that  semester’s  group  of  students  in  the  TICCIT  section  passing  the  GE  exam.  At  the  end 
of  the  4th  semester  93%  of  this  new  group  of  TICCIT  students  passed  the  two-part  GE  exam.  See  Eigure  3. 


The  J-curve  of  implementation  in 
TICCITEnglish  - semester4 


EIGURE  3.  TICCIT  design  experiment  results  over  four  semesters. 
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Conclusion 

In  this  paper,  we  have  shown  that  principled  design  experiments  are  necessary  to  understand  the  levels  of 
difficulty  in  a particular  domain  (area  or  subject)  and  also  what  those  levels  entail  in  terms  of  their  constructs 
(milestones,  benchmarks  or  key  tasks).  Because  principled  design  experiments  take  place  over  multiple  cycles,  and 
the  results  of  previous  cycles  serve  as  controls  for  the  next  cycle,  this  method  reduces  the  need  for  control  groups  as 
past  cycles  now  serve  as  the  “control”  with  which  future  measurements  can  be  compared.  This  method  also 
eliminates  the  false  assumption  of  equality,  which  happens  when  learning  experiments  attempt  to  utilize 
randomization.  Principled  design  experiments  know  exactly  where  the  learners  are,  without  having  to  guess  or 
assume  their  current  status. 

We  have  shown  that  once  a domain  is  delineated  (a  particular  subject  or  area  is  understood  in  terms  of  its 
levels  and  what  those  levels  entail  in  terms  of  their  key  tasks),  we  can  measure  progress,  give  meaningful  feedback, 
and  instruct  how  to  rapidly  progress  from  lower  levels  to  higher  levels  in  the  given  category  (domain).  Principled 
design  experiments  increase  performance  within  a domain  of  learning  by  providing  meaningful  instruction, 
evaluation,  and  feedback.  The  instruction,  evaluations,  and  feedback  are  all  linked  (connected)  to  the  particular  key 
task  or  milestone  at  a given  level  in  the  domain.  These  milestones  are  either  easy,  somewhere  between  easy  and 
hard,  or  hard. 

In  order  to  provide  meaningful  instruction,  evaluation,  and  feedback,  and  ultimately  to  rapidly  increase 
performance,  it  becomes  necessary  to  develop  “construct-linked”  scales  of  measurement.  These  scales  of 
measurement  are  connected  to  the  clearly  delineated  milestones  or  benchmarks  (constructs).  Instruction  and 
feedback  that  is  linked  or  connected  to  all  the  skills  in  these  construct -linked  scales  is  meaningful  and  useful,  and 
makes  for  efficient  progress  by  eliminating  unnecessary  activities  that  are  not  tied  to  the  construct  linked  scales  and 
ultimately  have  no  effect  on  an  individual’s  progress  within  the  domain. 

We  have  shown  that  in  order  for  construct-linked  (benchmark-connected)  scales  to  be  useful,  they  must  not 
vary  or  change  in  certain  ways.  We  identified  key  types  of  invariance  such  as  construct  and  invariance.  Finally,  we 
identified  a phenomena  known  as  the  7-curve  of  implementation,  which  commonly  occurs  as  we  introduce 
principled  designed  experiments.  As  progress  initially  dips  after  the  designed  experiments,  it  will  usually  far  surpass 
initial  learning  in  an  efficient  fashion.  Thus,  although  a dip  in  performance  may  initially  be  observed,  principled 
design  experiments  with  their  resultant  delineated  domains  and  construct-linked  invariant  scales  of  measurement, 
prove  to  be  a very  powerful  and  effective  way  to  increase  performance  and  progress  in  a rapid  and  efficient  manner. 
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Abstract 

This  study  investigated  the  comparative  effects  of  adjunct  questions,  student  self  generated  questions,  and 
note  taking  on  learning  from  a multimedia  database.  High  school  students  worked  individually  or  in  cooperative 
dyads  on  a computer-based  multimedia  unit  using  a study  guide  to  answer  either  adjunct  questions,  generate  self- 
questions,  or  take  notes  to  accomplish  a learning  task  focusing  on  the  events  of  the  civil  rights  movement  in  the 
United  States. 

One  hundred  seven  students  attending  a high  school  in  the  Southwest  were  the  participants  for  this  study. 
Prior  to  the  beginning  of  the  study,  each  student  was  randomly  assigned  to  one  of  the  six  treatment  groups. 
Materials  in  this  study  included  a multimedia  database  and  a Student  Packet , which  included  a study  guide  listing 
relevant  resources  to  review  and  a questioning  or  note-taking  strategy  to  assist  students  in  learning  the  material 
from  the  database. 

There  were  two  criterion  measures  in  this  study,  a posttest  and  an  attitude  survey.  Other  data  collected 
included  informal  observations,  student  interviews  and  review  of  the  student  study  guides. 

The  dependent  variable  was  achievement  as  determined  by  posttest  score.  Analyses  were  conducted  using 
both  traditional  Analysis  of  Variance  (AN OVA)  techniques  as  well  as  hierarchical  linear  modeling  (HUM).  Results 
of  the  ANOVA  indicated  that  students  in  the  note-taking  treatment  condition  performed  significantly  better  than 
students  working  in  the  self-generated  question  treatment  condition.  However,  because  of  the  dependency  of  scores 
of  students  working  in  pairs,  it  was  determined  that  HUM  techniques  were  more  appropriate.  Results  of  these 
analyses  did  not  yield  significant  differences  in  posttest  scores  among  the  treatment  conditions,  although  the 
correlations  of  posttest  scores  of  students  working  in  pairs  seemed  to  indicate  possible  levels  of  cooperation. 

Student  interview  responses  and  review  of  student  study  guides  seemed  to  provide  indications  of  how  students 
utilized  the  questioning  or  note-taking  strategies  while  working  in  this  environment.  Implications  for  the  design  of 
instruction  are  discussed. 

Introduction 

The  explosive  growth  in  computers  and  related  technologies  has  led  to  the  development  of  a variety  of 
innovative  learning  approaches.  One  aspect  of  this  technological  advancement  has  focused  on  an  increased  use  of 
open-ended  learning  environments  (OELEs).  Incorporated  in  many  of  these  environments  is  a focus  on  providing 
the  learner  with  control  of  the  learning  process  (Hannafin  & Land,  1997).  The  responsibilities  of  the  learner  often 
include  asking  relevant  questions,  pursuing  needed  knowledge,  and  evaluating  the  learning  experience.  These  types 
of  learning  processes  require  the  learner  to  use  various  cognitive  and  metacognitive  strategies  (Hannafin,  Hill,  & 
Land,  1997). 

Results  of  previous  research  have  suggested  that  some  learners  may  meet  the  cognitive  and  metacognitive 
demands  of  learning  in  these  types  of  environments,  however,  many  do  not  (Hannafin,  Hill  & Land,  1997;  Land, 
2000).  Very  often,  OELEs  are  developed  with  a variety  of  tools  and  resources  the  learner  could  use  to  facilitate  and 
enhance  the  learning  process.  Regardless  of  the  apparent  power  and  usefulness  of  the  tools,  however,  it  is  unlikely 
that  learners  will  automatically  use  them  if  they  do  not  understand  how  they  can  be  of  assistance  (Land  & Hannafin, 
1996;  Scardamalia,  Bereiter,  McLean,  Swallow  & Woodruff,  1989).  Some  learners  continue  to  rely  on,  or  even  to 
prefer,  externally -directed  methods  for  leaning  (Land  & Hannafin,  1996),  therefore,  prompting  them  can  perhaps 
induce  greater  reflection  on  the  learning  process  and  determination  of  their  own  level  of  comprehension  (Brush  & 
Saye,  2000;  Pressley,  1995). 

Student  monitoring  of  their  learning  and  comprehension  can  be  facilitated  by  utilizing  various  techniques 
during  the  learning  process.  In  their  review  of  the  research  on  comprehension  monitoring  strategies.  Dole,  Duffy, 
Roehler,  and  Pearson  (1991)  indicated  that  students  who  are  asked  to  determine  importance,  summarize  information. 
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draw  inferences,  and  answer  teacher-generated  as  well  as  student-generated  questions  recall  and  retain  more 
information  from  text  passages.  Therefore,  to  assist  students  in  learning  more  effectively  in  these  types  of 
environments,  various  learning  and  instructional  strategies  may  be  employed.  The  learning  strategies  that  may  be 
effective  include  adjunct  questions,  student  self-generated  questions  and  note-taking  and  the  instructional  strategy  of 
grouping. 

Adjunct  Questions 

Within  open-ended  learning  environments,  it  is  necessary  for  students  to  be  actively  engaged  in  their 
learning.  In  order  to  accomplish  learning  activities,  students  need  to  analyze  the  learning  task,  then  determine  the 
manner  in  which  they  will  accomplish  the  task  (Brush  & Saye,  2000).  Since  many  students  are  not  able  to 
successfully  manage  these  responsibilities,  learning  can  be  facilitated  by  the  use  of  adjunct  questions  (Hamilton, 
1992).  Adjunct  questions  are  usually  found  in  textbooks  following  text  passages  or  are  developed  by  teachers  to 
focus  students  on  a specific  learning  task  (Andre  & Anderson,  1978-1979). 

In  a review  of  the  research,  Hamaker  (1986)  analyzed  61  published  experiments  that  focused  on  the  effects 
of  adjunct  questions  on  comprehension.  Findings  from  this  review  indicated  that  inserted  postquestions  seem  to 
enhance  comprehension  by  facilitating  the  mental  processing  of  text  information.  Postquestions  seem  to  enable  the 
learner  to  develop  a sense  of  attention  to  the  particular  type  of  information  being  questioned.  This  increase  in 
attention  appears  to  influence  performance  on  all  types  of  test  questions,  even  those  not  directly  related  to  the 
adjunct  questions. 

Student  Self-generated  Questions 

Though  results  of  many  studies  have  shown  that  providing  students  with  adjunct  questions  to  answer  while 
reading  text  can  improve  their  recall  of  information,  development  of  appropriate  questions  may  be  dependent  upon 
extensive  work  done  by  instructors  prior  to  instruction.  But  where  such  teacher-generated  questions  are  not 
available,  students  can  still  monitor  their  learning  and  comprehension  through  the  use  of  self-questioning  (Andre  & 
Anderson,  1978-1979).  During  self-questioning,  learners  can  monitor  their  understanding  of  the  lesson  content  by 
asking  and  answering  questions  for  themselves  (King,  1991). 

Generating  self-questions  while  reading  appears  to  have  an  influential  effect  on  comprehension  due  to  the 
activation  of  various  cognitive  processes.  In  order  to  construct  questions,  students  must  encode  all  of  the  textual 
information  to  make  a decision  about  what  is  and  what  is  not  question-worthy  information  (Frase  & Schwartz, 

1975).  This  entails  a deeper  semantic  analysis  of  the  text  than  simply  reading  the  text  (Andre  & Anderson,  1978- 
1979).  By  posing  questions  for  themselves,  it  is  necessary  for  students  to  control  their  cognitive  processes  such  as 
focusing  attention  on  the  content  to  be  learned,  and  determining  the  organization  of  ideas  (King,  1991). 

There  has  been  considerable  research  on  the  effects  of  self-questioning  on  student  comprehension.  In  her 
review  of  27  studies  on  the  use  of  self-questioning  as  a reading  comprehension  strategy,  Wong  (1985)  found  that 
students  using  self-questioning  during  reading  generally  comprehended  more  than  did  students  who  used  other 
strategies. 

Note  Taking 

Because  multimedia  databases  use  ill-structured  design,  students  working  with  these  do  not  simply  respond 
to  the  systems,  but  become  an  integral  part  of  them  (Land  & Hannafin,  1996).  While  working  in  these  environments, 
students  need  to  determine  what  information  is  relevant  and  necessary  to  successfully  achieving  the  learning  goal. 

One  strategy  that  has  been  found  to  assist  students  in  selecting  important  information  from  instructional 
materials  is  note  taking.  Note  taking  is  an  activity  that  is  almost  automatic  for  many  students  since  it  is  so  prevalent. 
There  has  been  considerable  research  focusing  on  the  effects  of  note  taking  on  student  learning  (Fisher  & Harris, 
1973;  Kiewra,  1989;  Kulhavy,  Dyer,  & Silver,  1975).  Results  of  these  studies  seem  to  indicate  that  during  note 
taking,  students  are  actively  processing  the  learning  material.  Through  this  active  engagement  with  the  learning 
material,  students  are  better  able  to  encode  new  information.  The  encoding  function  of  note  taking  suggests  that 
through  the  process  of  recording  notes,  students  may  have  increased  attention,  they  may  elaborate  more  on  specific 
ideas,  and  may  even  reorganize  new  information  and  put  it  into  their  own  words  (Fisher  & Harris,  1973;  Kiewra, 
1989). 

Cooperative  Learning 

While  utilization  of  these  learning  strategies  may  effectively  improve  learning,  social  interaction  with 
others  can  also  be  valuable.  Individually  and  socially  mediated  monitoring  are  complementary  and  are  both 
important  for  helping  students  learn  to  reflect  upon  their  learning  (Lin,  Hmelo,  Kinzer  & Secules,  1999).  Often  the 
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reflection  process  can  be  improved  by  comparing  and  contrasting  one’s  learning  to  others.  In  this  sense,  social 
interactions  may  point  to  new  ways  of  thinking  about  learning  (Johnson  & Johnson,  1986).  Cooperative  learning 
techniques  are  typically  used  in  education  to  encourage  this  type  of  social  interaction. 

Despite  considerable  research  that  has  been  conducted  on  the  effects  of  these  strategies,  there  are  still  gaps 
in  our  knowledge.  Still  to  be  investigated  are  the  comparative  effects  of  adjunct  questions,  student  self-generated 
questions,  note  taking  and  grouping  on  student  achievement  in  multimedia-based  learning  environments. 

Purpose  of  the  Study 

The  purpose  of  this  study  was  to  investigate  the  effects  of  question  type  (adjunct  questions,  student  self- 
generated questions  and  note  taking/control)  and  grouping  strategy  (individuals  and  pairs)  on  learning  from  a 
multimedia  database. 

Method 

Participants 

One  hundred  seven  students  attending  a high  school  in  the  southwest  were  the  participants  in  this  study. 
There  were  57  male  and  50  female  students.  All  participants  were  enrolled  in  an  American  Government  class. 

Materials 

Materials  in  this  study  included  Decision  Point!,  a multimedia  database  relating  to  the  African-American 
civil  rights  movement  (Brush  & Saye,  2000;  Saye  & Brush,  1999).  The  database  featured  primary  print  documents 
in  the  form  of  newspaper  and  magazine  articles,  political  and  editorial  cartoons,  and  personal  accounts  as  well  as 
video  new  footage,  interviews,  and  music.  Students  were  also  provided  with  a Student  Packet  that  consisted  of  a task 
sheet  providing  brief  background  information  about  the  lesson  content,  a study  guide  listing  relevant  resources  and  a 
questioning  or  note-taking  strategy,  instructions  for  completing  the  lesson  using  the  study  guide,  and  a job  aid  to 
assist  in  navigation  of  the  database. 

Procedures 

The  study  took  place  during  regular  class  periods  over  three  consecutive  days.  During  the  first  two  class 
periods,  students  worked  with  the  Decision  Point!  database  completing  their  study  guides.  On  the  third  day,  students 
completed  the  posttest  and  attitude  survey.  Each  class  period  lasted  52  minutes. 

Prior  to  the  beginning  of  the  study,  each  student  was  randomly  assigned  to  one  of  the  six  treatment  groups.  Students 
who  were  assigned  to  one  of  the  three  cooperative  dyad  treatment  conditions  were  then  randomly  assigned  to  a 
partner.  Students  in  the  cooperative  dyads  worked  with  the  same  partner  during  the  two  class  sessions  using  the 
database. 

Criterion  Measures 

There  were  two  criterion  measures  in  this  study,  a posttest  and  an  attitude  survey.  The  posttest  was  a paper- 
and-pencil  instrument  designed  to  measure  student  achievement  on  the  lesson  task  using  the  Decision  Point! 
database.  The  posttest  was  worth  a total  of  16  points  and  included  13  multiple -choice  items,  worth  one  point  each, 
and  one  short  answer  item,  worth  three  points. 

A 12-item  attitude  survey  was  administered  following  the  posttest.  The  first  ten  items  were  Likert -type 
items  using  a four-point  scale  ranging  from  Strongly  Agree  (4)  to  Strongly  Disagree  (1),  and  the  last  two  items  were 
open-ended  response  type  items.  The  ten  Likert-items  were  based  on  four  themes;  two  questions  asked  students  to 
rate  their  preference  for  working  with  a partner  or  working  alone,  four  questions  asked  students  to  rate  their 
preference  for  working  with  computer-based  lessons,  three  questions  asked  students  to  rate  the  impact  the  study 
guide  had  on  their  learning  of  the  material,  and  one  item  asked  students  to  rate  their  effort  in  working  through  this 
lesson.  The  open-ended  response  items  asked  students  to  provide  at  least  one  thing  that  they  liked  the  most  about  the 
lesson  and  one  thing  that  they  liked  the  least  about  the  lesson. 

Other  data  collected  included  informal  observations,  student  interviews  and  review  of  the  student  study 
guides.  Informal  observations  were  conducted  during  the  two  data  collection  days.  Student  questions  and  comments 
regarding  the  lesson  were  recorded.  After  completing  the  lesson  posttest  and  attitude  surveys,  two  students  from 
each  treatment  condition  were  randomly  selected  to  participate  in  an  interview  with  the  researcher.  The  interview 
followed  a scripted  protocol.  Student  Study  Guides  were  also  collected  and  reviewed  by  the  researcher  to  determine 
whether  students  followed  specific  treatment  directions  in  working  through  the  lesson. 

Design  and  Data  Analysis 
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The  experimental  design  for  this  study  was  a posttest-only,  control  group  design  with  random  assignment 
of  students  to  treatments.  It  was  a three  (adjunct  questions,  self-generated  questions,  note  taking/control)  by  two 
(individual,  informal  cooperative  dyad)  factorial  design.  The  dependent  variable  was  achievement  as  determined  by 
posttest  score.  Analyses  were  initially  conducted  using  Analysis  of  Variance  (ANOVA)  followed  by  Hierarchical 
Linear  Modeling  (HLM). 

It  is  expected  that  posttest  scores  of  students  working  in  cooperative  pairs  may  be  more  similar  than  scores 
of  students  between  pairs.  The  similarity  may  indicate  a correlation  or  dependency  in  the  scores.  Analysis  did 
indicate  that  dependency  did  exist  between  the  scores  of  students  within  the  cooperative  pairs.  Traditional  ANOVA 
is  not  robust  to  a violation  of  the  dependency  assumption,  consequently,  analysis  of  variance  using  HLM  was 
conducted. 

Hierarchical  linear  modeling  is  similar  to  traditional  ANOVA  in  that  the  major  interest  is  in  determining 
main  and  interaction  effects.  HLM  directly  assesses  the  correlation  of  scores  of  students  within  cooperative  pairs  and 
takes  that  into  consideration  in  determining  significant  main  and  interaction  effects.  These  correlations  provide 
interpretable  information  and  may  indicate  cooperation  between  students  working  in  pairs.  A positive  correlation 
may  indicate  cooperation  between  students  working  in  pairs;  zero  correlation  may  indicate  lack  of  cooperation  or 
indifference  between  students  working  in  pairs;  and  a negative  correlation  may  perhaps  indicate  competition 
between  students  working  in  pairs.  The  result  of  HLM  analysis  is  likely  to  differ  from  traditional  ANOVA  to  the 
extent  that  dependency  does  exist. 

The  HLM  model  used  to  analyze  the  achievement  data  was  the  least  restrictive,  in  that  it  allowed  the 
variances  and  covariances  to  vary  across  all  three  cooperative  groups.  This  model  also  allowed  for  different 
variances  across  all  six  treatment  conditions. 

Attitude  data  were  analyzed  using  both  analysis  of  variance  (ANOVA)  and  HLM.  For  simplicity,  the  same  HLM 
model  that  was  used  to  assess  the  two  main  and  interaction  effects  for  the  achievement  data  was  used  to  analyze  the 
attitude  data.  Open-ended  item  responses  were  analyzed  and  categorized  by  discernable  themes.  The  frequency  of 
responses  in  each  thematic  category  was  then  calculated. 

Notes  from  the  informal  observations  were  compiled  and  summarized.  Interview  question  responses  from  12 
students  were  summarized  by  treatment  condition. 

All  Study  Guides  were  reviewed  to  determine  whether  students  completed  the  lesson  following  the  specific 
directions  provided  in  their  Study  Guides.  Evaluation  was  based  on  the  appropriateness  of  the  responses,  questions 
generated,  and  notes  for  each  resource  reviewed.  For  the  Study  Guides  from  the  adjunct  question  treatment 
conditions,  the  student  responses  to  each  of  the  adjunct  questions  were  reviewed  to  determine  whether  the  response 
written  down  pertained  to  the  particular  resource.  For  the  Study  Guides  from  the  self-generated  question  treatment 
condition,  both  the  questions  and  responses  were  reviewed  to  determine  whether  the  question  and  response  were 
related  to  the  resource.  For  the  Study  Guides  from  note  taking  treatment  condition,  student  notes  were  reviewed  to 
determine  whether  the  notes  included  information  found  in  the  particular  resource. 


Results 

Results  are  reported  below  for  posttest  performance  and  review  of  the  student  study  guides  only.  Results 
from  informal  observations,  and  interview  responses  will  be  provided  in  the  full  research  report  to  be  published  at  a 
later  date. 

Posttest  Achievement 

The  means  and  standard  deviations  for  posttest  performance  as  a function  of  the  two  factors  are  presented 
in  Table  1.  In  the  sample,  individual  treatment  mean  posttest  scores  were  highest  for  students  in  the  adjunct  and 
note-taking  treatments.  The  mean  score  for  students  in  the  adjunct  treatment  was  10.84  (SD  = 2.52)  and  for  students 
in  the  note-taking  treatment  the  mean  score  was  10.63  (SD  = 2.59).  The  mean  score  for  students  in  the  self- 
generated question  treatment  was  9.41  (SD  = 2.85).  A similar  pattern  occurred  with  mean  posttest  scores  in  the 
sample  cooperative  dyad  treatment  conditions.  Again,  the  highest  mean  posttest  s cores  were  for  students 
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Table  1 Posttest  Means  and  Standard  Deviations  by  Question  Type  and  Grouping  Condition 


Adjunct  Question  Self -generated  Note  Taking  Total 

Question 


Individual 

M 

= 

10.84 

M 

= 

9.41 

M 

= 

10.63 

M 

= 

10.33 

SD 

= 

2.52 

SD 

= 

2.85 

SD 

= 

2.59 

SD 

= 

2.67 

n 

= 

19 

n 

= 

17 

n 

= 

19 

n 

= 

55 

Informal 

M 

= 

10.22 

M 

= 

9.25 

M 

= 

11.44 

M 

= 

10.35 

Cooperative  Dyad 

SD 

= 

3.32 

SD 

= 

3.53 

SD 

= 

2.23 

SD 

= 

3.13 

n 

= 

18 

n 

= 

16 

n 

= 

18 

n 

= 

52 

Total 

M 

10.54 

M 

9.33 

M 

11.03 

M 

10.34 

SD 

= 

2.91 

SD 

= 

3.15 

SD 

= 

2.42 

SD 

= 

2.89 

n 

= 

37 

n 

= 

33 

n 

= 

37 

n 

= 

107 

Note.  Maximum  score  = 16. 


participating  in  the  adjunct  and  note-taking  treatments.  The  mean  score  for  students  in  the  adjunct  treatment  was 
10.22  (SD  = 3.32)  and  for  students  in  the  note-taking  treatment,  the  mean  score  was  1 1.44  (SD  = 2.23).  The  mean 
score  for  students  in  the  self-generated  question  treatment  condition  was  9.25  (SD  = 3.53). 

A 3 X 2 traditional  Analysis  of  Variance  (ANOVA)  and  Hierarchical  Linear  Modeling  (HLM)  were 
conducted  to  evaluate  the  effects  of  the  three  question  type  conditions  and  grouping  on  posttest  performance.  Table 
2 provides  an  ANOVA  summary  table  for  posttest  performance  scores.  The  traditional  ANOVA  indicated  a 
significant  main  effect  for  question  type,  F(2,  101)  = 3.25,  p = .04,  partial  = .06,  but  no  significant  effect  for 
grouping,  F(l,  101)  = 0.00, p =.98,  partial  = 0.00  and  no  significant  interaction  between  question  type  and 

grouping,  F(2,  101)  = .61, p = .55,  partial  = .01. 


Table  2 ANOVA  Summary  Table  for  Posttest  Achievement  Scores 


Source 

df 

F 

Partial 

P 

Question  Type 

2 

3.25* 

.06 

.04 

Grouping 

1 

0.00 

0.00 

.98 

Question  Type  by 
Grouping 

2 

.61 

.01 

.55 

Note:  < .05. 


Follow-up  tests  were  conducted  to  evaluate  pairwise  differences  among  the  means.  Because  the  sample 
variances  among  the  three  groups  ranged  from  4.97  to  12.46,  it  was  assumed  that  the  variances  were  not 
homogeneous  and  post  hoc  comparisons  using  the  Dunnett’s  C test  were  conducted.  Dunnett’s  C is  a test  that  does 
not  assume  equal  variances  among  the  groups.  Results  indicated  that  there  were  significant  differences  in  the  means 
between  the  note-taking  treatment  condition  and  the  self-generated  question  treatment  condition  at  the  .05  level.  The 
students  in  the  note-taking  treatment  condition  earned  a significantly  higher  posttest  mean  score  than  did  students  in 
the  self-generated  question  treatment  condition.  The  results  of  these  tests,  as  well  as  the  means  and  standard 
deviations  for  the  question  type  treatments,  are  reported  in  Table  3. 
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Table  3 Differences  Among  Posttest  Scores  for  the  Question  Type  Treatment  Conditions 


Question  Type 

M 

SD 

Adjunct 

Question 

Self -generated 
Question 

Adjunct  Question 

10.54 

2.91 

Self -generated 
Question 

9.33 

3.15 

NS 

Note  Taking 

11.03 

2.42 

NS 

* 

Note.  NS  = nonsignificant  differences  between  pairs  of  means,  while  an  asterisk  (*)  = significance  using  the 
Dunnett’s  C procedure,  p < .05. 

Posttest  scores  between  students  working  in  pairs  may  be  similar  and  results  from  ANOVA  may  not  be 
reliable  in  these  situations.  Though  much  of  the  previous  research  investigating  the  effects  of  cooperative  grouping 
on  learning  has  been  analyzed  using  ANOVA  techniques,  it  was  determined  that  hierarchical  linear  modeling  may 
be  a more  appropriate  type  of  analysis  to  utilize  (S.  Green,  personal  communication,  February,  2002).  While  the 
results  of  the  HLM  analyses  are  interesting,  they  should  be  viewed  with  caution  due  to  the  small  sample  size  of  this 
study. 

In  analyzing  the  data  for  the  current  study,  multiple  mixed  models  were  conducted.  All  models  included  the 
main  and  interaction  effects  for  question  type  and  grouping,  but  differed  in  the  estimation  of  variances  and 
covariances  within  cells.  The  most  complex  model  allowed  for  different  residual  variances  among  the  six  cells  and 
different  covariances  between  pairs  in  the  three  cooperative  learning  conditions. 

After  thorough  evaluation,  it  was  determined  that  the  first  model  should  be  considered  the  best  model  for 
the  data  since  it  was  the  most  conservative,  allowing  for  the  fewest  assumptions.  In  this  model,  the  variances  and 
covariances  were  allowed  to  vary.  Based  on  this,  results  of  hierarchical  linear  modeling  indicated  no  significant 
main  effect  for  question  type,  F{2,  75)  = 2.60, p = .08,  no  significant  main  effect  for  grouping,  F(l,75)  = 0.00,  p = 
.99,  and  no  significant  interaction  between  question  type  and  grouping,  F{2,  75)  = .61,  p = .55.  Results  of  this 
analysis  are  presented  in  Table  4. 

Table  4 FILM  Summary  Table  for  Posttest  Achievement  Scores 


Source 


df 


F 


P 


Question  Type 

2 

2.60 

.08 

Grouping 

1 

0.00 

.99 

Question  Type  by 
Grouping 

2 

.61 

.55 

Note:  *p  < .05. 

Review  of  Student  Study  Guides 

All  of  the  student  study  guides  were  reviewed  to  determine  whether  students  used  the  question  or  note 
taking  strategy  provided  in  their  study  guides,  and  for  indications  as  to  how  they  may  have  used  the  particular 
strategy.  Review  of  the  study  guides  for  the  adjunct  treatment  condition  found  that  students  answered  the  questions, 
though  the  thoroughness  of  the  written  responses  varied.  Some  students  provided  short,  one  sentence  answers  and 
others  wrote  out  lengthy  responses  including  several  short  paragraphs.  Study  guides  of  students  participating  in  the 
adjunct  treatment  cooperative  dyad  treatment  condition  revealed  mostly  identical  responses  written  by  both 
members  of  the  dyads. 
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Review  of  the  study  guides  for  students  in  the  self-generated  question  treatment  condition  found  that 
students  generally  used  the  study  question  starters  provided.  Most  of  the  questions  they  wrote,  however,  were  not 
related  to  the  lesson  objective,  but  focused  instead  on  unimportant  details  found  in  the  resource  documents.  Study 
guides  for  students  participating  in  the  cooperative  self-generated  question  treatment  condition  included  questions 
created  using  the  study  question  starters,  but  again,  many  questions  were  unrelated  to  the  lesson  objective.  The 
questions  and  responses  written  by  students  within  the  dyads  were  typically  identical. 

Review  of  the  study  guides  for  students  in  both  the  individual  and  cooperative  note -taking  treatments  found  lists  of 
bulleted  items,  matrices  of  information,  columns  of  information  with  headings,  and  arrows  pointing  from  one  note  to 
another.  For  many  students  in  the  cooperative  dyads,  the  notes  recorded  by  one  student  were  not  the  same  as  those 
recorded  by  their  partner,  and  sometimes  the  amount  of  information  recorded  varied  as  well.  Overall  in  comparison, 
it  appeared  that  generally  students  in  this  treatment  condition  wrote  down  more  information  than  students  in  the  two 
question-type  treatment  conditions. 


Discussion 

Overall  Achievement 

Results  for  achievement  indicated  that  variations  in  question  type  and  grouping  strategy  yielded  similar 
effects  on  student  performance  on  the  unit  posttest.  The  level  of  student  achievement  was  fairly  low  and  there  were 
not  significant  differences  in  scores  among  students  in  the  six  treatment  conditions.  Nevertheless,  the  results  of  this 
investigation  provide  important  findings  useful  for  improving  learning  in  nultimedia-based  environments. 

Overall,  there  seem  to  be  several  factors  that  may  have  contributed  to  the  general  tendency  of  these  results. 
During  the  design  of  this  investigation,  conversations  with  the  high-school  teachers  revealed  that  most  of  their 
instruction  was  expository  in  nature.  Students  had  not  used  any  type  of  computer-based  instruction  (CBI)  in  their 
social  studies  courses  and  it  was  unlikely  that  they  had  used  CBI  in  any  of  their  other  high-school  courses.  Students 
participating  in  the  study  were  also  not  familiar  with  the  events  of  the  civil  rights  movement  in  the  United  States. 

The  content  information  and  lesson  included  as  part  of  this  investigation  were  to  serve  as  an  introduction  for  their 
instructional  unit  on  civil  rights,  which  was  to  begin  the  week  following  data  collection.  Therefore,  both  the 
Decision  RomC  database  and  the  events  of  the  civil  rights  movement  were  unfamiliar  to  the  students.  The  novelty  of 
the  learning  experience  in  terms  of  both  multimedia -based  instruction  and  lesson  content  may  have  limited  student 
achievement  levels.  Other  researchers  have  noted  the  importance  of  these  two  elements  in  influencing  learning  from 
multimedia  learning  environments  (Hill  & Hannafin,  1997;  Land  & Greene,  2000). 

Another  factor  that  may  have  contributed  to  the  low  achievement  scores  may  be  the  students’  lack  of  skill 
in  critical  reasoning  about  social  issues.  From  their  previous  social  studies  classes,  students  were  accustomed  to 
expository  instruction  in  which  they  were  told  what  was  imp  ortant  to  learn.  During  data  collection,  several  students 
commented  “How  do  I know  what  is  important?’’  and  “All  the  documents  are  saying  the  same  thing.”  Questions  and 
comments  such  as  these  seem  to  indicate  students’  unfamiliarity  in  working  not  only  in  an  ill -structured  learning 
environment  but  also  with  an  unstructured  subject  domain  such  as  social  studies  (Scheurman  & Newmann,  1998; 
Wineburg,  1991).  These  findings  are  similar  to  those  from  previous  research  utilizing  the  Decision  Fo/nt.' database 
(Brush  & Saye,  2000;  Saye  & Brush,  1999).  In  their  investigations,  students  also  complained  about  repetition  in  the 
database  documents  and  the  lack  of  capsule  summaries.  These  researchers  suggested  that  students  were  confused  by 
the  messiness  of  the  raw  data  and  were  not  adequately  prepared  to  interpret,  synthesize,  and  evaluate  the  primary 
documents  found  in  the  database.  Brush  and  Saye  (2000)  suggest  that  in  order  for  students  to  work  more 
successfully  in  these  types  of  environments,  they  need  more  experiences  in  problem-based  discovery  methods  and 
perhaps  more  structured  support  throughout  the  learning  process. 

Individual  versus  Cooperative  Grouping 

Achievement  as  indicated  by  posttest  scores  of  students  participating  in  the  cooperative  dyads  was  similar 
to  the  achievement  of  students  in  the  individual  treatments.  Other  researchers  investigating  the  impact  of  cooperative 
grouping  on  learning  in  computer-based  environments,  too,  have  not  always  found  significant  differences  between 
individuals  and  cooperative  groups  (Cavalier  & Klein,  1998;  Crooks,  et  al.,  1996;  Crooks,  et  ah,  1998). 

There  may  be  several  possible  reasons  that  cooperative  grouping  did  not  have  significant  effects  on 
learning  in  this  study.  Previous  research  on  cooperative  grouping  has  outlined  components  critical  to  the 
effectiveness  of  this  instructional  strategy  in  computer  based  learning  environments  (Johnson  & Johnson,  1986; 
Johnson  & Johnson,  1996;  Johnson  et  al,  1986).  These  components  include  positive  interdependence,  individual 
accountability,  shared  leadership,  shared  responsibility  for  each  other’s  learning,  and  the  processing  of  group 
effectiveness.  The  absence  of  any  these  elements  may  prevent  the  beneficial  effects  of  cooperative  learning. 
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Random  assignment  was  used  to  compose  the  cooperative  dyads.  Because  of  this,  students  may  have  been 
working  with  a partner  that  they  did  not  know  prior  to  this  lesson.  This  unfamiliarity  may  have  inhibited  the  manner 
in  which  students  worked  together.  Even  though  students  were  given  detailed  instruction  sheets  describing  how  they 
should  work  with  their  partner  to  review  the  various  database  resources  utilizing  their  particular  questioning  or  note- 
taking strategy,  the  instructions  may  have  been  deficient  in  providing  explanation  of  the  other  elements  critical  to 
effective  cooperative  learning.  In  a review  of  research  on  the  effectiveness  of  cooperative  grouping  on  achievement 
in  computer-based  instruction,  Susman  (1998)  suggests  that  students  may  not  truly  understand  how  to  work  with  a 
partner  or  may  be  lacking  in  the  skills  necessary  to  interact  effectively.  Her  review  found  that  when  researchers 
included  training  on  the  key  elements  of  cooperative  learning,  student  achievement  was  higher. 

The  present  study  represents  a departure  from  previous  methodologies  in  that  this  study  compared  the 
effects  of  question  type  and  note  taking  on  student  achievement  on  a multimedia-based  lesson.  Researchers  in  earlier 
studies  have  often  compared  the  effects  of  a particular  learning  strategy  condition  to  those  of  a control  group 
condition  not  utilizing  a learning  strategy.  For  example,  in  previous  research  focusing  on  the  effects  of  adjunct 
questions  on  student  learning,  many  of  the  studies  finding  significant  effects  compared  adjunct  questioning 
treatments  to  control  groups  receiving  no  questions  (Hamaker,  1986).  The  current  study,  however,  provides  insight 
into  the  effectiveness  of  three  various  strategies  on  student  learning  in  a multimedia  environment. 

Adjunct  Questions 

In  the  current  study,  the  adjunct  question  study  guide  provided  students  with  a resource  to  review  and  a 
related  question  to  answer.  From  the  format  of  the  study  guide,  students  were  able  to  read  the  adjunct  question  prior 
to  reading  the  related  document.  Student  responses  to  the  interview  question  asking  how  they  utilized  this 
questioning  strategy  indicated  that  they  would  read  the  question  first  then  search  the  document  for  the  answer. 
Previous  research  on  adjunct  questions  has  suggested  that  when  allowed  to  review  the  text  after  encountering  the 
question,  students  are  less  motivated  to  study  the  entire  text,  but  process  the  text  superficially,  searching  for  the 
response  to  the  question  (Hamaker,  1986). 

In  the  Brush  and  Saye  (2000)  investigations  described  earlier,  categorizing  questions  were  provided  within 
the  Decision  Point!  database  to  help  students  organize  and  synthesize  information  about  a particular  event.  Students 
initially  ignored  the  questions,  and  even  after  being  prompted  to  use  them,  they  simply  filled  in  the  spaces  without 
thoughtful  reflection  of  the  material  reviewed  and  the  possible  connections  to  the  learning  goal  (Brush  & Saye, 

2000;  Saye  & Brush,  1999).  The  lack  of  requisite  content  knowledge  may  have  inhibited  the  effectiveness  of  the 
adjunct  questions. 

The  amount  of  prior  knowledge  a student  has  can  affect  his  or  her  ability  to  learn  new  information.  Without 
prior  knowledge,  it  may  be  difficult  for  students  to  understand  the  significance  of  new  material,  to  select  main  ideas, 
and  to  disregard  unimportant  details  (Brown,  Campione,  & Day,  1981).  In  order  to  comprehend  new  information,  it 
is  necessary  for  students  to  actively  generate  relationships  between  the  new  information  and  previous  knowledge 
and  experiences  (Shuell,  1988;  Wittrock,  1990). 

While  the  effects  of  the  adjunct  questions  on  learning  were  similar  for  both  the  individual  and  cooperative 
grouping  treatments,  the  results  indicate  other  interesting  findings.  Analysis  of  the  posttest  scores  for  the  cooperative 
dyads  utilizing  the  adjunct  questioning  strategy  indicated  a fairly  strong  correlation  between  the  scores  of  the 
individuals  within  the  dyads.  Review  of  the  student  study  guides  found  that  student  responses  to  the  adjunct 
questions  were  similar  in  some  cases  but  identical  in  most.  This  may  indicate  that  students  worked  together  to 
formulate  the  answers  to  the  questions. 

Answering  questions  perhaps  promotes  more  student  interaction,  thereby  facilitating  learning.  This  finding 
is  similar  to  those  of  previous  research  studies  focusing  on  student  interaction  in  cooperative  groups  (Webb,  1982a; 
Webb,  1982b).  Results  from  Webb’s  studies  indicated  that  giving  and  receiving  explanations  had  a positive  effect  on 
learning.  By  giving  explanations,  students  need  to  create  verbal  associations  from  the  information  being  reviewed 
and  may  involve  generating  new  elaborations.  Receiving  information  from  another  student  may  enhance  learning 
because  students  tend  to  use  language  that  other  students  understand  and  tend  to  recognize  each  other’s  nonverbal 
cues  about  whether  they  comprehend  the  material. 

Other  research  has  found  that  generating  solutions  to  questions  embedded  in  a lesson  and  explaining  how  a 
solution  was  determined  was  positively  and  significantly  correlated  with  success  (Hooper,  1992).  Students  who 
generate  solutions  to  questions  and  who  form  mental  elaborations  by  explaining  lesson  concepts  to  their  partners 
may  process  lesson  content  at  a deeper  level  and  demonstrate  higher  achievement  than  would  more  passive  students. 
An  important  goal,  therefore,  may  be  to  promote  active  learning  in  cooperative  groups^ 
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Student  Self-generated  Questions 

Much  of  the  previous  research  on  student  self-generated  questions  has  focused  on  the  effects  of  student 
learning  using  self-generated  questions  from  text  -based  lessons  (Andre  & Anderson,  1978-1979;  Frase  & Schwartz, 
1975;  Wong  & Jones,  1982).  Other  research  has  focused  on  the  effects  of  student  self-generated  questions  on  lecture 
comprehension  (King,  1989).  The  use  of  student  self-generated  questions  to  enhance  learning  in  a multimedia 
environment  is  a new  consideration. 

The  creation  of  questions  while  working  through  a lesson  was  a novel  learning  strategy  for  the  students  in 
the  current  study.  Students  participating  in  this  treatment  condition  were  given  a study  guide  indicating  relevant 
resources  to  review  along  with  ten  possible  question  starters  and  detailed  written  explanations  for  generating 
questions. 

Review  of  the  student  study  guides  indicated  that  students  used  the  question  starters  that  were  provided. 
Many  of  their  questions,  however,  focused  on  unimportant  details  found  in  the  database  documents,  and  not 
information  that  was  relevant  to  the  lesson  objective  - to  determine  where  the  civil  rights  activists  were  more 
successful  at  achieving  their  goals.  Some  examples  of  student  generated  questions  included,  “What  is  the 
significance  of  June  13,  1963?”,  “Compare  and  contrast  the  NAACP  with  SNCC.”,  and  “How  many  people  were 
arrested  on  April  6?”  The  inability  of  students  to  generate  questions  focused  on  the  lesson  objective  may  also  be  due 
to  lack  of  prior  content  knowledge. 

Lack  of  prior  knowledge  may  lessen  the  students’  ability  to  generate  topic -related  questions  (Wong,  1985). 
To  be  able  to  ask  a question,  a student  needs  to  have  an  optimal  amount  of  prior  knowledge  for  the  particular  subject 
matter.  This  may  be  particularly  important  when  working  in  a multimedia  database  because  of  the  unstructured 
nature  of  this  type  of  learning  environment  and  the  vast  amount  of  resources  available  to  the  student  (Greene,  1995; 
Saye  & Brush,  1999).  Generating  questions  does  not  directly  lead  to  comprehension  of  information,  but  the  process 
of  generating  appropriate  questions  and  searching  through  the  database  for  answers  may  help  students  combine  new 
information  with  their  prior  subject  knowledge  (Rosens hine,  Meister,  & Chapman,  1996). 

Generating  questions  that  are  either  too  broad  or  too  narrow  has  been  described  as  “question  drift”  (Land, 
2000;  Land  & Greene,  2000).  Question  drift  exists  when  a student  asks  a question  and  then  attempts  to  find 
informatbn  that  has  something  to  do  with  that  topic.  If  the  student  does  not  find  information  related  to  their 
question,  they  may  change  directions  and  reformulate  their  question  to  match  the  information  they  have  found. 

Similar  to  the  findings  from  those  in  the  adjunct  treatment  condition,  analysis  of  the  posttest  scores  for  the 
cooperative  dyads  utilizing  the  self-generated  questioning  strategy  indicated  a fairly  strong  correlation  between  the 
scores  of  the  individuals  within  the  dyads.  Review  of  the  student  study  guides  indicated  that  both  the  questions  and 
the  responses  for  each  student  in  the  pair  were  typically  identical.  Again,  this  seems  to  indicate  that  students  worked 
together  to  create  the  questions  and  the  answers.  Similar  to  the  findings  for  the  individual  treatment  condition,  the 
students  used  the  question  starters,  however,  many  of  their  questions  focused  on  unimportant  details  found  in  the 
database  documents,  and  not  information  that  was  relevant  to  the  lesson  objective. 

Though  other  researchers  have  found  student  self-generated  questions  to  have  a positive  effect  on  the 
posttest  performance  of  high-school  students  (King,  1991;  Wong,  1985),  many  of  these  studies  included  extensive 
self-questioning  training  programs.  In  the  current  study,  however,  students  were  successful  in  using  the  question 
starters  to  generate  questions.  Their  inability  to  generate  questions  that  focused  on  the  lesson  objective  may  be 
attributable  to  the  students’  lack  of  prior  knowledge  and  their  inexperience  in  interpreting  content  information  that  is 
presented  in  an  unstructured  manner.  This  seems  to  indicate  that  students  primarily  need  more  support  in 
interpretation  of  original  source  documents,  and  secondarily  support  in  generating  questions  relative  to  the  goal  of 
the  lesson. 

Note  Taking 

In  the  current  study,  students  in  the  note-taking  treatment  achieved  the  highest  posttest  scores,  though  the 
differences  were  not  significant.  Students  in  the  note-taking  treatment  were  given  a study  guide  indicating  relevant 
resources  to  review.  A blank  area  was  provided  after  each  resource  listed  for  students  to  record  important 
information. 

Results  from  previous  research  indicate  that  note  taking  can  be  an  effective  study  strategy  (Dyer,  Riley  & 
Yekovich,  1979;  Kulhavy,  Dyer  & Silver,  1975).  Findings  from  these  studies  have  shown  that  students  taking  notes 
while  reading  were  able  to  focus  their  attention  in  a selective  manner  on  certain  parts  of  the  material.  Through  the 
process  of  note  taking,  students  are  able  to  reorganize  the  data  and  put  it  into  their  own  words.  This  transformation 
process  appears  to  assist  students  in  the  encoding  process  that  is  critical  to  effective  learning  (Fisher  & Harris, 

1973). 
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Review  of  the  student  study  guides  from  the  note-taking  treatment  produced  several  interesting  findings. 
When  recording  information,  many  students  created  bulleted  lists,  and  organized  information  into  columns  with 
headings.  Other  students  created  diagrams  to  organize  material  found  in  the  database  and  some  used  arrows  which 
seemed  to  indicate  cause-effect  relationships  between  situations  described  in  the  documents.  The  use  of  these  types 
of  elaboration  and  organizational  techniques  may  enhance  student  learning  (Weinstein,  1988).  By  paraphrasing  and 
creating  schemes  to  organize  information,  students  are  actively  involved  in  their  attempt  to  learn  the  material. 

Adding  some  sort  of  symbolic  construction  to  new  information  seems  to  make  information  more  meaningful  to  the 
student  and  may  enhance  learning  and  recall  of  information. 

Another  advantage  of  note  taking  as  a study  strategy  seems  to  result  from  the  time  students  spend  studying 
the  material  (Kulhavy,  Dyer  & Silver,  1975).  Results  from  previous  research  found  a positive  relationship  between 
achievement  of  students  taking  notes  and  study  time.  The  researchers  speculated  that  the  high  performance  of 
students  taking  notes  may  be  due  to  the  fact  that  they  spent  more  time  with  the  material.  Though  time  on  task  was 
not  tracked  in  the  current  study,  review  of  the  student  study  guides  in  the  note-taking  treatment  shows  a quantitative 
difference  from  the  other  two  treatments  in  the  amount  of  information  written  down.  Since  students  in  the  note- 
taking treatment  recorded  more  information  than  those  students  in  either  of  the  other  two  questioning  treatments, 
perhaps  they  read  more  of  the  document  which  may  have  facilitated  their  learning.  Student  responses  to  the 
interview  question  asking  them  how  they  utilized  the  note-taking  strategy  found  that  all  of  them  read  the  document 
while  writing  down  information  that  they  found  to  be  important. 

For  many  students,  note  taking  is  a study  strategy  that  they  are  very  familiar  with  and  commonly  use.  In  a 
previous  study  investigating  the  study  strategies  used  by  students  in  a computer  literacy  course,  students  were  asked 
to  describe  the  study  strategy  they  used  the  most  for  learning  the  material  in  that  course  and  all  of  their  other 
courses.  The  majority  of  students  indicated  that  in  both  situations  reading  the  textbook  and  taking  notes  was  their 
most  preferred  method  of  study  (Niemczyk  & Savenye,  2001).  In  the  current  study,  in  response  to  the  interview 
question  asking  students  how  they  learn  the  best,  several  students  stated  that  taking  notes  was  their  most  effective 
learning  strategy. 

In  contrast  with  the  results  found  in  the  adjunct  question  and  self-generated  question  treatment  conditions, 
posttest  scores  for  the  individuals  within  the  dyads  in  the  note-taking  treatment  had  a negative  relationship.  Within 
some  of  the  dyads,  the  posttest  score  for  one  student  was  high,  while  the  posttest  score  for  the  other  student  was  low. 

Review  of  the  study  guides  for  the  students  participating  in  the  cooperative  dyad  treatment  condition  found, 
that  for  some  dyads,  the  information  written  down  was  similar,  but  for  others,  it  was  different.  This  seems  to  indicate 
that  students  were  writing  down  information  that  they  determined  was  personally  important  to  them.  The  lack  of 
similarity  also  seems  to  indicate  that  students  in  this  treatment  condition  may  not  have  had  as  much  interaction  as 
did  students  in  the  other  two  treatments. 

Another  interesting  finding  from  review  of  the  study  guides  revealed  that  for  some  dyads  the  amount  of 
information  written  down  appeared  to  diminish  as  the  students  progressed  through  the  study  guide.  Within  these 
dyads,  the  study  guide  for  one  student  contains  a comprehensive  set  of  notes,  while  the  other  student’s  notes  seem  to 
decrease  and  even  stop  in  some  cases.  This  may  be  attributable  to  students’  lack  of  skill  and  understanding  of  the 
elements  necessary  to  effective  learning  in  cooperative  groups.  Previous  research  has  yielded  similar  findings  in  that 
one  student  may  interact  directly  with  the  computer  while  the  other  student  watches  (Susman,  1998).  When  one 
student  is  interacting  with  the  computer  and  the  other  student  is  watching,  they  are  not  acting  as  a team  and  may 
therefore  not  receive  the  benefits  of  cooperative  learning.  For  learning  to  be  successful,  students  need  to  be  active  in 
the  processing  of  information  (Brown,  Collins  & Duguid,  1989;  Shuell,  1988). 

Implications 

The  results  of  this  study  have  implications  for  the  design  of  instruction  using  multimedia  databases. 

Though  previous  research  has  found  adjunct  questions,  student  self-generated  questions  and  note  taking  to  be 
effective  strategies  in  text  and  lecture-based  lessons,  they  may  not  be  adequate  in  assisting  student  learning  when 
working  with  multimedia  databases.  Because  of  the  unstructured  nature  of  multimedia  databases,  students  may  need 
more  explicit  instructional  support  in  problem  solving  and  critical  thinking  skills  (Saye  & Brush,  1999).  It  may  also 
be  necessary  to  provide  students  support  in  monitoring  their  learning  and  understanding  when  working  in  a complex 
environment.  Managing  these  processes  can  be  especially  challenging  for  learners  with  little  prior  content 
knowledge  and  lack  of  experience  in  working  with  unstructured  learning  environments  (Hannafin,  Hill  & Land, 

1997;  Land,  2000). 

By  design,  multimedia  databases  provide  students  with  vast  amounts  of  resources  in  an  unstructured 
manner.  Because  of  the  open-endedness  of  the  learning  environment,  it  may  be  necessary  to  provide  students  with 
instructional  support  that  is  appropriate  for  their  particular  learning  situation,  making  the  need  for  the  support  to  be 
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dynamic.  It  may  therefore,  be  beneficial  to  incorporate  more  teacher-student  interactions  when  utilizing  these 
databases.  Other  researchers  (Land,  2000;  Brush  & Saye,  2000)  have  suggested  that  when  using  OELE’s,  teachers 
could  provide  students  with  guiding  questions  and  encourage  discussions  explicitly  aimed  at  promoting  deeper 
understanding  of  the  lesson  content. 

Conclusion 

As  the  incorporation  of  technology  and  multimedia  databases  continues  to  increase  in  use  in  classrooms,  it 
is  important  to  determine  how  they  might  be  used  most  effectively  in  enhancing  student  learning.  This  investigation 
was  the  first  study  to  focus  on  the  comparative  effects  of  adjunct  questions,  student  self-generated  questions  and 
note  taking  on  student  learning  in  these  environments.  It  was  also  determined  through  this  study  that  hierarchical 
linear  modeling  techniques  may  be  most  appropriate  in  analyzing  data  from  studies  focusing  on  the  effects  of 
cooperative  grouping.  Results  from  these  types  of  analyses  may  provide  indications  of  the  levels  of  cooperation 
among  students  working  in  cooperative  pairs.  It  is  important  that  researchers  consider  the  results  of  this  study  and 
continue  to  investigate  methods  that  may  assist  student  learning  from  multimedia-based  lessons. 
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Abstract 

Web-based  distance  education  has  evolved  as  one  of  the  most  innovative  and  potentially  rewarding 
segments  of  the  higher  education  market.  Among  its  attractions  are  the  opportunities  it  provides  academic 
administrators  to  use  faculty  in  ways  that  reach  new  learner  markets,  overcome  constraints  of  time  or  space,  and 
capitalize  on  evolving  learning  technologies.  For  administrators  to  be  effective  in  the  new  higher  education 
environment,  they  must  continually  monitor,  among  other  things,  changes  occurring  among  faculty  involved  in 
distance  education.  In  particular,  as  institutions  continue  to  expand  their  curricula  to  distance  learners,  they 
should,  likewise,  assess  faculty  adaptation  to  challenges  of  teaching  online.  Such  assessments  enable 
administrators  to  identify  gaps  between  where  the  faculty  is,  and  where  it  ought  to  be,  all  things  considered. 
Furthermore,  the  assessments  provide  valuable  input  in  managing  faculty  involvement  in  distance  learning  and 
promoting  overall  institutional  success  in  this  area. 

Introduction 

In  the  last  two  decades,  many  educational  institutions  have  actively  sought  to  integrate  computing 
technologies  into  the  classroom  in  part  because  the  society  demanded  it,  and  students  expected  to  learn  with  it.  The 
World  Wide  Web  (the  Web)  in  particular,  has  increasingly  become  a principal  medium  of  reaching  students  in 
remote  locations.  The  strength  of  the  Web  lies  in  its  ability  to  promote  interactivity,  a key  component  in  the 
transmission  of  knowledge.  Thus,  while  the  educational  media  of  the  past  demanded  the  passive  attention  of  the 
user,  the  Web  requires  active  participation.  The  appeal  of  the  Web  for  learning  also  lies  in  its  other  characteristics 
such  as  unlimited  distribution  capacity  and  relatively  inexpensive  production  capabilities. 

Historically,  technology  has  often  been  perceived  as  a burden  rather  than  as  a facilitator  of  teaching  and 
learning.  This  has  led  to  skepticism  and  disagreements  about  the  need  and  role  of  technology  in  teaching  and 
learning.  On  one  hand,  some  scholars  have  lamented  that  instructors  are  continually  required  to  train  just  to  master 
new  technology,  maintain  it  and  become  fa  miliar  with  how  to  incorporate  it  effectively  in  the  classroom.  Such 
training,  they  argue,  has  taken  away  faculty  commitment  of  time  and  effort  in  mastering  their  respective  disciplines 
and  contributing  to  the  respective  knowledge  bases.  The  faculty,  in  other  words,  finds  itself  caught  in  the  position  of 
dual  pressures  from  the  discipline  and  the  technology.  At  the  same  time,  it  has  been  argued  that  many  institutions 
lack  focus  in  the  professional  development  opportunities  they  provide  to  faculty  who  need  to  learn  new  technologies 
or  upgrade  existing  ones. 

Today,  the  number  of  institutions  of  higher  education  engaged  in  distance  learning  continues  to  grow. 

Many  are  also  investing  heavily  in  computing  infrastructure  (Massy  & Zemsky,  1996)  to  support  their  growing 
computing  initiatives.  Among  the  challenges  that  are  facing  these  institutions  is  that  the  faculty  continue  to  question 
the  value  of  technology  in  teaching  and  learning.  Research  shows  that  faculty  often  view  technology  as  too 
impersonal  and  a threat  to  the  quality  of  instruction  (Rice  & Miller,  2001).  The  truthfulness  of  this  perception  is, 
however,  suspect.  Although  traditional  methods  of  teaching  such  as  the  lecture  often  fail  to  promote  closer  contact 
between  instructors  and  students  (Oblinger,  1995),  it  is  assumed  that  newer  modes  of  instructor  delivery  could  only 
be  worse.  To  add  to  the  skepticism  about  technology,  a study  of  248  research  reports  from  1982-1996,  (Russell, 
1996)  found  no  significant  differences  between  the  learning  outcomes  achieved  through  traditional  instruction  and 
those  obtained  through  a variety  of  distance  learning  technologies  such  as  teleconferencing,  videotape,  satellite  and 
computer-assisted  instruction  (Rice  & Miller,  2001).  Technology  enhanced  distance  education,  nonetheless, 
remains  a powerful  mode  of  reaching  non-traditional  learners  who  cannot  attend  the  traditional  classroom  but  need 
alternative  means  to  fulfill  their  educational  needs.  The  faculty,  again,  is  implicitly  expected  to  respond  to  that  need 
by  developing  even  more  distance  learning  courses. 

This  case  study  investigates  the  needs,  concerns,  self-assessments  and  other  perceptions  of  faculty  who, 
with  limited  or  no  prior  experience  in  using  the  Web  for  instruction,  taught  Web-based  distance  education  courses. 
The  study  first  looks  at  the  above  issues  from  the  perspectives  of  individual  faculty  members.  It  then  looks  at  the 
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same  issues  from  the  faculty  as  a group.  The  study  finally  contrasts  faculty  perceptions  with  those  of  the 
administrator  in  charge  of  the  academic  unit  (school)  from  which  the  participants  were  chosen. 

Context  of  the  Case 

In  the  fall  of  1999,  a southern  medium-sized,  historically  black  university  decided  to  extend  its  distance- 
learning program  by  offering  courses  through  the  World  Wide  Web.  The  institution  was  committed,  by  its  land  grant 
status,  to  providing  broad  access  to  higher  education.  One  unit  of  the  university,  the  School  of  Technology,  was 
particularly  considered  a pioneer  in  distance  learning.  It  had  been  offering  classes  through  its  extension  programs  in 
satellite  campuses  since  the  early  1980s.  Faculty  at  this  School  were  eager  and  willing  to  try  a new  way  of  reaching 
students  who  would  otherwise  not  be  able  to  attend  classes  on  campus.  The  school  had  a ready  client  base  already 
in  place.  Hence,  the  faculty  easily  focused  on  developing  and  conducting  distance-education  classes  via  the  Web. 

The  client  base  mainly  comprised  of  high  school  teachers  who  needed  to  update  their  subject  matter  knowledge, 
pursue  graduate  degrees  or  renew  credentials  for  certification.  All  faculty  members  who  wished  to  teach  a Web- 
based  course  were  invited  to  do  so.  Only  five  agreed  to  experiment  with  the  Web-based  distance  learning.  These 
were  seen  as  the  innovators  and  early  adopters  of  Web-based  distance  education. 

This  research  was  carried  out,  with  the  cooperation  of  the  dean  of  the  School,  in  order  to  learn  about  the 
experiences  of  select  faculty  members  as  they  embarked  on  a new  way  of  delivering  instruction,  and  resolving  the 
related  challenges.  He  supported  the  study  on  the  belief  that  it  would  provide  valuable  information  for  administrators 
who  must  determine  how  to  motivate  and  support  faculty  who  choose  to  integrate  technology  in  their  teaching. 

Research  Questions 

This  study  sought  to  answer  three  main  research  questions:  (a)  How  did  faculty  perceive  their  Web-based 
distance  learning  needs?  (b)  How  did  faculty  perceive  their  capacity  to  conduct  Web-based  distance  learning?  And 
(c)  What  were  the  key  faculty  concerns  in  adopting  Web-based  distance  learning? 

Review  of  the  Literature 

Most  studies  in  distance  learning  focus  on  students’  perceptions  and  attitudes  (Freitas,  Myers,  & Avtgis, 
1998),  technologies  (Saba,  1999;  Smith  & Dillon,  1999),  and  fiscal  implications  (Feenberg,  1999).  The  experiences 
of  the  of  the  instructors  have  not  been  thoroughly  examined,  therefore  there  is  a need  for  studies  that  focus  more  on 
this  area.  Faculty  perceptions  of  teaching  technologies  are  influenced  by  the  perceived  educational  effectiveness  of 
the  technologies,  their  ease  of  use,  personal  orientation,  and  the  institutional  environment.  In  order  enhance  the 
learning  outcomes  for  students,  higher  education  institutions  must  ultimately  resolve  the  issue  of  how  faculty  adapt 
to  and  use  educational  technologies.  Much  of  the  literature  on  educational  technology  adoption  by  the  faculty 
features  early  adopters  more  than  any  other  group.  Geoghegan  (1995),  states  that  these  individuals  (a)  favor 
revolutionary  change  and  are  (b)  visionaries;  (c)  strong  in  their  technology  focus;  (d)  risk  takers;  (e)  experimenters, 
and,  (f)  largely  self-sufficient.  Their  perceptions  of  distance  learning,  however  are  also,  influenced  by  the  larger 
environment  in  which  they  work. 

Higher  education  is  becoming  more  competitive.  Member  institutions  are  paying  keener  attention  to  the 
changing  demographics  that  show  large  numbers  of  older  students,  displaced  homemakers,  single  parents  and  people 
in  need  of  continuing  education  opportunities.  These  individuals,  also  referred  to  as  non-traditional  students,  wish  to 
further  their  personal  and/or  career  educational  goals  without  the  time  restrains  of  class  schedules.  Higher  education 
institutions  have  responded  by  launching  courses  for  delivery  at  a distance  with  the  help  of  new  communication 
technologies  such  as  the  Web  (Hanna,  1999,  p.  19). 

Web-based  instruction  (WBI)  offers  a new  way  of  exploiting  current  technologies  and  shifting  higher 
education  from  an  instructor-centered  model  to  a learner-centered  model.  WBI,  however,  is  only  a vehicle  for 
providing  a learning  environment  and  delivering  knowledge.  Although  it  has  the  potential  to  revolutionize  how 
learning  takes  place,  not  all  students  or  courses  are  suited  for  this  mode  of  instruction  (Clark,  1994). 

The  adoption  of  new  learning  technologies  in  teaching  and  learning  presents  new  challenges  and  concerns 
for  faculty.  Some  of  the  most  frequently  mentioned  challenges  are:  decreased  live  face-to-face  interaction  with 
students  (Berge,  1998;  Clay,  1999);  lack  of  time  to  plan  and  deliver  courses  online  (Berge,  1998;  Clay,  1999);  and 
the  lack  of  support  in  planning  and  delivering  courses  online  (Berge,  1998;  Clay,  1999).  Other  concerns  include 
time  investment  by  faculty  members  to  learn  the  medium  (Rockwell,  Schauer,  Fritz  & Marx,  1999);  inadequate 
compensation  and  incentives  (Rockwell,  Scheuer,  Fritz  & Marx,  1999);  a heavier  workload  (Rockwell,  Scheuer, 

Fritz  & Marx,  1999)  and  slow  computer  access.  Earlier  studies  found  similar  findings  (Dillon,  1989).  Now,  over  a 
decade  later,  many  institutions  of  higher  education  are  still  struggling  to  integrate  and  utilize  distance  education 
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technologies  (Dooley  & Murphrey,  2000).  Wolcott  reported  that  rapid  technological  change  will  ''dramatically 
reshape  faculty  members'  roles  and  their  work  environment,"  and  that  the  faculty  reward  system  will  have  to 
consider  innovativeness  and  technological  change  (Wolcott,  1997,  p.  3).  "The  view  of  distance  education  as  an 
innovation  provides  the  impetus  for  researching  the  phenomena,  particularly  from  the  perspective  of  those  upon 
whom  its  acceptance  depends:  the  faculty"  (Dillon  & Walsh,  1992,  p.  6). 

The  preceding  review  suggests  that  research  concerning  faculty  perceptions,  although  crucial  to 
understanding  the  diffusion  or  adoption  of  distance  learning  technologies  in  institutions  of  higher  learning,  is  very 
limited.  Hence,  there  is  a need  for  exploration  of  just  how  the  faculty  perceive  this  phenomenon  and  more 
significantly,  how  they  assess  the  pedagogical  soundness  of  this  medium  (Reeves  & Reeves,  1997). 

Methods 

Participants 

The  participants  included  the  dean  and  five  faculty  members  from  The  School  of  Technology.  The  faculty 
members  were,  at  the  time,  novices  at  teaching  Web-based  distance  learning  courses.  The  faculty  members  were 
familiar  with  the  online  course  delivery  platform  known  as  e-College.  They  were  selected  through  an  initial 
interview  with  the  dean  of  the  school. 

Data  Collection 

The  qualitative  data  gathering  techniques  used  in  this  study  included  interviews  with  individual 
participants,  an  online  survey  consisting  of  open-ended  questions,  electronic  artifacts  and  a focus  group.  All  the 
interviews  were  semi -formal.  This  approach  was  employed  to  provide  flexibility  and  overcome  time  constraints  of 
the  research  . 

The  interviews  centered  around  the  following  questions:  (a)  How  did  you  become  involved  in  distance 
learning?  (b)  What  do  you  perceive  to  be  the  limitations  of  Web-based  distance  learning?  (c)  How  do  you  perceive 
your  capacity  to  conduct  a Web-based  distance  learning  course?  (d)  What  administrative  policies  would  help  you  to 
become  an  effective  Web-based  distance  learning  instructor?  The  interview  protocol  was  designed  with  the 
assistance  of  an  interdisciplinary  team  of  experts.  Specifically,  a distance  learning  administrator,  an  instructional 
designer,  and  a specialist  in  interviewing  procedures  provided  input  in  the  design,  testing  and  revision  of  the 
protocol.  They  also  oversaw  the  interviewing  process.  The  first  interview  was  conducted  with  the  administrator 
(dean).  This  approach  provided  the  researchers  with  an  ’’empowered”  entry  into  the  school  and  helped  them  to 
understand  the  background  of  its  culture.  Data  from  the  preliminary  interviews  were  used  in  shaping  the  online 
survey  instrument.  This  instrument  had  four  sections:  a)  questions  dealing  with  perceptions  of  WBI  in  general;  (b) 
statements  dealing  with  faculty  capacity  to  use  technology;  (c)  questions  dealing  with  non-technical  concerns  and  d) 
questions  about  demographics.  A focus  group  was  used  to  encourage  participants  to  interact  with  one  another  in 
articulating  concerns  and  perceptions  of  WBI.  This  meeting  was  recorded  on  audiocassette,  then  transcribed  word- 
for-word.  During  the  focus  group,  the  moderator  asked  the  participants  probing  questions  in  an  effort  to  reveal 
subtle  sentiments.  The  moderator  sometimes  repeated  questions  to  triangulate  results  and  verify  information. 

During  each  aspect  of  the  study,  the  researchers  followed  the  predefined  protocols  as  much  as  possible.  In 
addition,  they  kept  reflexive  notes,  extensive  audit  trails  of  correspondence,  interview  schedules,  logistical 
information,  insights,  methodological  decisions  and  participant  codes  and  other  documentation. 

Data  Analysis 

Substantive  analysis  of  the  collected  data  involved  sorting,  summarizing,  consolidating,  interpreting  and 
identifying  themes  and  characteristics.  An  independent  reader  was  included  in  the  analytical  process  as  a 
precaution,  to  assure  objective  interpretation  of  the  data.  During  the  analysis,  all  responses  corresponding  to  each 
question  were  summarized  to  provide  a holistic  and  accurate  view  of  each  content  area.  In  addition,  individual  items 
were  coded  and  categorized  into  groups  of  similar  answers.  The  categorized  data  were  then  analyzed  for  specific 
patterns  and  compared  across  questions  to  identify  consistent,  repetitive  threads  in  quality  and  content.  Stable  or 
persistent  patterns  were  noted  and  reported. 
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Findings 

General  Findings 

The  interview  with  the  administrator  (dean)  revealed  fundamental  background  of  the  school.  It  also  put  in 
perspective,  how  the  participants  and  the  school  and  the  institution  came  to  be  involved  in  distance  learning. 

In  the  process  of  data  collection,  it  became  clear  that  faculty  had  difficulty  getting  students  to  participate  in  online 
chats  or  threaded  discussions.  Further,  some  of  the  faculty  did  not  believe  in  synchronous  chats  because  they  were 
viewed  as  hindering  the  flexibility  of  class  participation  which  distance  education  is  meant  to  promote.  It  was 
therefore  impossible  for  the  researchers  to  observe  virtual  chat  sessions  as  previously  planned.  The  faculty  members 
and  the  administrator  perceived  Web-based  distance  learning  as  potentially  more  effective  than  the  traditional 
classroom  in  reaching  diverse  segments  of  learners.  This  view  was  aligned  with  one  of  the  institutional  goals,  which 
was,  to  provide  convenient  access  to  education.  Faculty  members  were  concerned  about  the  additional  time  required 
to  develop  and  implement  online  courses.  They  wanted  explicit  incentives  to  offset  the  commitments  of  their  time. 

One  individual’s  reaction  to  the  time  demands  summed  the  general  sentiment  of  the  faculty  on  this  issue: 

“. . .Personally,  I bit  off  more  than  I could  chew. . .in  terms  of  the  number  of  courses  I am  teaching. . .but  I also  bit  off 
more  than  I could  chew  in  departmental  service,  university  service,  and  professional  service.  And  so  I didn’t  enjoy 
the  course  like  I know  I could  enjoy  it. . .but  if  I had  a regular  load,  or  whatever,  I bet  I would  have  enjoyed  it”. 

Faculty  members  felt  that  their  continued  success  as  distance  learning  providers  had  to  be  balanced  by  the 
following  incentives:  monetary  compensation,  lower  course  loads,  release  time,  release  from  other  institutional 
responsibilities,  support  of  colleagues,  and  institutional  support  through  teaching  assistants.  Faculty  members,  for 
the  most  part,  did  not  perceive  the  smaller  class  sizes  that  they  had  in  distance  learning  courses  as  significant 
inducement  to  continue  teaching  via  the  Web.  They  were  quite  satisfied  with  the  training  and  technological  support 
that  they  were  getting,  but  strongly  felt  that  they  needed  more  support  especially  from  peers  to  continue  their 
involvement  in  distance  learning.  They  also  expressed  the  need  for  policy  changes  at  departmental,  and 
institutional  levels.  These  are  addressed  in  detail  under  the  relevant  individual  questions  that  were  posed  in  the 
study. 

Findings  About  Specific  Questions 

The  responses  to  the  question,  “How  did  you  become  involved  in  distance  learning?”  indicated  that 
individual  faculty  interest  in  exploring  new  ways  of  teaching  was  the  single  common  deciding  factor.  Although  the 
opportunity  to  teach  was  extended  to  all  faculty  members  in  the  school,  only  a few  accepted  the  challenge.  There 
were  salient  differences  in  the  perception  of  the  faculty  and  the  administrator  about  the  reasons  for  engaging  in 
distance  learning.  These  reasons  are  shown  in  Table  1. 

Table  1:  Faculty  and  Administrator  Reasons  for  Engaging  in  Distance  Learning 


Reason 

Respondent 

Convenience  to  both  faculty  and  students 

Faculty  and  Administrator 

Ability  to  serve  more  students 

Faculty  and  Administrator 

Increased  flexibility  in  working  hours  and  location 

Faculty 

Reach  a wider  range  of  students 

Faculty  and  Administrator 

New  avenue  to  compete  for  students 

Administrator 

Add  value  and  prestige,  particularly  in  the  eyes  of  accrediting  bodies 

Administrator 

The  responses  to  the  question,  “ What  do  you  perceive  to  be  the  limitations  of  Web-based  distance 
learning?”  indicated  that  faculty  concerns  about  adopting  Web-based  distance  learning  were  frequently  linked  to  the 
current  limitations  of  the  technology.  Despite  these  limitations  the  faculty  members  were  unanimously  optimistic 
about  the  future  of  distance  learning.  In  addition,  they  expected  continuous  growth  and  institutional  support.  The 
faculty,  however,  remained  skeptical  about  the  suitability  of  the  Web  for  delivering  certain  types  of  courses.  Table  2 
represents  details  of  the  perceived  limitations  of  distance  learning. 
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Table  2:  Faculty  Perceptions  of  the  Limitations  Of  Distance  Learning 


Limitation 

Impact  on  learning 

Implications  for  course  delivery 

Lack  of  tools  to 
support  hands- 
on  tasks 

Students  are  limited  to  learning 
only  the  theoretical  aspects  of  the 
discipline. 

Since  certain  courses  are  unsuitable  for  online  delivery 
(e.g.  courses  with  laboratory  components),  students  would 
have  to  attend  the  traditional  classroom  to  meet  some  of 
their  degree  requirements.  Courses  requiring  hands-on 
experience  are  less  likely  to  be  considered  for  delivery  on 
the  Web. 

Oratory  and 
presentation 
skills 

Lack  of  opportunity  to 
demonstrate  practical  skills  (e.g. 
classroom  teaching;  theatrical 
presentations. 

Such  courses  should  not  be  offered  online  since  it  would 
be  difficult  to  assess  some  core  requirements  in  an  online 
environment. 

Physical 

isolation 

Limited  socialization,  even  when 
online  discussions/chats  are  used. 

Alternative  methods  for  promoting  a sense  of  community 
may  be  inconvenient  to  the  distance  learner. 

Engaged 
supervision  by 
instructor 

Lack  of  self-motivation  can 
impede  progress  for  a distance 
learner. 

Dropout  rate  can  be  high. 

Develop  strategies  to  keep  learners  motivated.  Perpetual 
office  hours  are  already  burdensome  to  the  instructor 
Administrator  favored  graduate  over  undergraduate 
courses. 

Technology  is 
not  transparent 
to  the  learner 

Concentration  shifts  from  learning 
to  mastering  technology. 
Fmstration  if  technology  hinders 
learning. 

Course  progress  is  hindered  by  the  adverse  feelings  that 
students  may  have  for  the  technology. 

The  responses  to  the  question,  “How  do  faculty  perceive  their  capacity  to  conduct  Web-based  distance 
learning?'*  indicated  that  faculty  members  in  general,  did  not  find  technology  complex;  all  of  them  easily  adapted  to 
it.  One  participant  summed  up  the  technology  challenge  as  follows:  “The  problem  is  not  technology;  it  is  the  lack  of 
time  to  use  it  to  its  fullest’’.  While  the  majority  had  adequate  technology  resources  and  were  very  comfortable  using 
them,  they  identified  several  other  problems  related  to  technology  that  threatened  their  success  in  online  learning. 
These  are  summarized  in  Table  3. 


Table  3:  Faculty  Perceptions  of  Technology  Challenges  or  Barriers  and  Remedies  to  Distance  Learning 


Technology  Challenges/Barriers 

Remedy /Comments 

Student  proficiency  in  using  technology 
(the  Web,  email,  FTP,  etc.). 

Mandate  student  orientation  prior  to  taking  an  online  class. 

Threaded  discussions,  e-mail  and  chat  do 
not  replace  the  face-to-face  interaction. 
The  nuances  of  speech,  facial  expressions 
are  absent.  Everything  can  be  taken 
literally  in  that  environment,  and,  at  times, 
misinterpreted. 

While  the  entire  faculty  acknowledged  the  importance 
communication  in  the  online  classroom,  the  majority  of  them 
perceived  synchronous  chat  as  an  unfavorable  requirement  in  DL 
because  it  requires  that  all  students  meet  at  a designated  time.  This, 
in  a sense,  imposes  class  time  on  students  who  wanted  to  have  a 
flexible  schedule. 

System  failure. 

Select  an  online  platform  that  is  reliable  and  responsive  to  ensure 
continuous  service. 

Timed  logouts  that  are  inconsistent  with 
course  tasks. 

Select  an  online  platform  that  is  flexible  and  adapts  to  evolving  user 
needs. 

The  responses  to  the  question,  “What  administrative  policies  are  necessary  for  you  to  become  an  effective 
Web-based  distance  learning  instructor?’’  indicated  that  faculty  responses  often  differed  from  those  of  the 
administrator.  They  ranged  from  compensation  to  increased  support  for  developing  online  courses.  The  faculty 
perceived  monetary  compensation,  as  a very  important  issue  while  the  administrator  did  not,  citing  release  time  and 
course  load  reduction  as  the  most  important  incentives. 

Although  none  of  the  faculty  had  large  online  classes,  most  felt  that  as  the  popularity  of  online  classes 
grew,  it  was  important  to  have  a policy  in  place  limiting  the  number  of  students  per  class  to  assure  quality  in 
instruction.  One  participant  suggested  a class  size  of  15  while  another  suggested  an  even  lower  number.  The  latter 
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found  it  difficult  to  manage  an  online  class  with  only  a few  students.  He,  therefore,  favored  further  reduction  in 
future  class  sizes.  The  faculty  members  unanimously  agreed  that  online  classes  required  a lot  more  time  to  manage 
than  they  had  anticipated.  They  also  agreed  that  they  needed  relief  from  institutional  activities  as  well  as  reduced 
course  loads  in  order  to  have  more  time  to  develop  and  teach  online  courses.  The  proposed  limit  on  the  class  size 
may  have  been  in  direct  conflict  with  the  broader  institutional  goal,  which  was,  to  increase  enrollment  through 
distance  learning  programs.  Paradoxically,  faculty  who  were  adamant  about  limiting  class  size  had  smaller  classes 
than  those  who  had  larger  ones.  A summary  of  the  findings  follows  in  Table  4. 


Table  4: 

Faculty  Ranking  of  Administrative  or  Policy  Changes  Required  to  Enhance  Distance  Learning 

Rank 

Suggestions  for  Administrative/Policy  Changes 

1 (Top) 

Release  time 

2 

Compensation  (monetary,  load  reduction) 

3 

Limit  of  class  size 

4 

Enforcement  of  rules  and  consequences  (registration,  communication  and  observation  of  deadlines) 

5 

Need  for  more  support  to  develop  new  online  courses 

6 

Need  for  continued  support  after  courses  are  developed 

7 

Need  for  technical  resources 

Discussion  and  Implications 

This  study  examined  faculty  perceptions  of  Web-based  distance  learning  and  offers  insights  of  how  one 
institution’s  distance  learning  program  should  be  approached  by  administrators,  faculty  and  support  staff  in  order  to 
succeed.  Although  the  nature  of  qualitative  research  impedes  the  generalization  of  the  findings  to  a larger 
population,  the  study  presents  some  challenges  about  distance  learning  that  may  be  universal.  These  challenges 
ought  to  be  considered  to  improve  distance  learning  at  least  in  the  institution  in  which  the  research  was  conducted. 
The  role  of  early  adopters  in  technology  adoption  can  be  pivotal  in  getting  other  faculty  involved  in  using 
technology  in  the  curriculum.  While  it  is  likely  that  early  adopters  may  continue  to  be  involved  in  Web-based 
distance  learning  in  spite  of  their  existing  concerns,  their  sustained  involvement  could  eventually  be  jeopardized  by 
a lack  of  interest  in  addressing  those  concerns.  Faculty  perceptions  of  distance  are  influenced  by  multiple  variables 
including  their  interaction  with  technology,  interaction  with  the  students,  interaction  with  the  support  systems  and 
their  own  experiences.  Administrators  should  make  every  effort  to  support  early  adopters,  as  a means  to  assure 
them  of  their  commitment  to  technology  integration  and  the  success  of  distance  learning  programs.  This  may  in  turn 
positively  influence  the  rate  of  adoption  of  future  instructional  technologies  not  only  among  early  adopters  but  also 
across  all  other  categories  of  potential  technology  users. 

Administrators  need  to  reexamine  how  the  introduction  of  new  technologies  impacts  faculty  workload  and 
the  other  structures  that  support  those  technologies.  Administrators  should  create  scenarios  that  encourage  faculty  to 
adopt  technology.  These  may  include  support  for  incentives  that  the  faculty  value,  continual  training,  support  for 
peer-led  information  sessions  and  presentations  by  experts  on  the  pedagogical  benefits  of  using  technology.  The 
level  of  commitment  to  technology  by  the  administrators  ultimately  determines  the  long-term  success  of  the 
adoption  of  new  technologies  by  the  faculty. 
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Introduction 

Recently,  much  attention  has  been  focused  on  identifying  and  developing  the  methods  to  use  the 
information  technologies  to  support  and  facilitate  learning  in  non-traditional  learning  environments.  Efforts  to  apply 
a paradigm  change  from  the  objectivist  view  of  learning  to  the  constructivist  view  of  learning  into  educational 
practice  lead  the  change  in  what  and  how  to  teach  in  school  and  learners’  role.  This  transition  can  be  accelerated  by 
recent  technological  developments.  In  fact,  information  and  communication  technologies  offer  transformed 
approaches  to  learning  and  teaching  as  well  as  opportunities  for  enhanced  and  more  flexible  access  to  information 
(Lajoe,  2000),  provide  the  new  ways  of  interaction  for  students,  especially  time  independent  and  place  independent 
way  of  communication,  and  thus  enable  learning  to  be  more  flexible  to  students  (Khan,  1997).  As  students  gain 
more  opportunity  to  learn  in  their  own  time  and  own  pace,  the  more  self-regulated  learning  ability  of  the  students 
comes  to  be  needed  (Ley  & Young,  1998).  This  study  focuses  on  how  to  support  learners  in  constructivist  learning 
environments  by  using  Web  technology  as  a cognitive  tool. 

Background 

As  we  move  further  into  the  knowledge  era,  what  to  teach  in  school  has  continued  to  evolve.  Unlike  the 
industrial  era  when  skills  needed  in  their  jobs  or  what  to  teach  in  school  were  relatively  standardized,  today’s 
employers  have  growing  demands  for  continually  customizing  skills.  People  must  be  able  to  seek  out,  analyze,  and 
use  information  by  themselves  to  adapt  themselves  to  the  ever-changing  role  of  their  jobs  as  the  amount  of 
information  continues  to  grow  at  an  exponential  rate.  The  goal  of  education  may,  therefore,  be  focused  more  than 
ever  on  the  acquisition  of  the  necessary  skills  so  that  workers  can  renew  their  knowledge  by  themselves.  To  meet 
the  needs  of  this  fast  changing  society,  learning  should  stretch  out  across  a lifetime  beyond  the  formal  education 
system  (Fisher,  in  press).  People  should,  therefore,  be  lifelong  learners.  Universities  as  a part  of  the  changing  world 
are  being  called  to  play  a critical  role  in  helping  students  to  become  lifelong  learners. 

How  to  meet  the  needs  of  the  lifelong  learner  is  an  area  in  need  of  further  exploration.  However,  some 
research  has  been  undertaken  that  gives  insight  into  what  is  needed  for  lifelong  learners.  One  of  the  most  important 
abilities  of  lifelong  learners  is  self-regulated  learning  ability.  Pintrich  (1995)  argued  that  lifelong  learners,  whether 
inside  or  outside  of  the  classroom,  self-regulate  their  own  learning.  Self-regulated  learning  is  known  to  contribute  to 
learning  achievement  and,  at  the  same  time,  it  can  be  the  outcome  of  the  learning  (Pintrich,  1995).  In  addition,  he 
argued  “it  is  not  a characteristic  that  is  genetically  based  or  formed  early  in  life’’  (p.  5).  That  means  self-regulated 
learning  can  be  improved  through  the  intentionally  designed  learning  experience.  Therefore,  self-regulated  learning 
has  gained  a great  deal  of  interest  among  academic  researchers  and  practicing  educators  because  it  appears  to  be  a 
worthy  objective  for  students  of  all  ages  in  all  disciplines  (Paris  & Paris,  2001). 

Parallel  with  this  societal  change  that  emphasizes  lifelong  learning,  the  paradigm  shift  from  an  objective 
view  of  learning  to  a constructivist  view  of  learning  accelerates  the  change  in  what  and  how  to  teach  in  school  and 
also  impact  the  learners’  role.  A discussion  about  the  constructivist  learning  environments  and  the  role  of  learners 
within  that  context  follows  in  the  next  section. 

Constructivist  Learning  Environments  and  the  Role  of  Learners 

Constructivist  learning  environments  put  learners  in  the  center  of  learning  and  emphasize  the  active  role  of 
learners  in  their  learning  (Duffy  & Cunningham,  1996).  To  be  more  successful  learners  in  the  constructivist 
learning  environments,  students  should  be  more  expert  self-regulated  learners. 

Jonassen  (1999)  distinguished  objectivist  conceptions  of  learning  and  constructivist  conceptions  of 
learning.  According  to  him,  “objective  conceptions  of  learning  assume  that  knowledge  can  be  transferred  from 
teachers  or  transmitted  by  technologies  and  acquired  by  learners’’  (p.  217).  On  the  other  hand,  “constructivist 
conceptions  of  learning  assume  that  knowledge  is  individually  constructed  and  socially  coconstructed  by  learners 
based  on  their  interpretations  of  experiences  in  the  world’’  (Jonassen,  1999,  p.  217).  With  the  advent  of  the 
constructivist  concepts  of  learning,  the  learning  is  in  transition  from  teacher-centered  to  learner-centered. 

A myriad  of  learning  theories  based  on  the  constructivist  view  of  learning  has  emerged  in  recent  years, 
including  open  learning  environments  (Hannafin,  Land,  & Oliver,  1999),  problem-based  learning  (Hmelo,  1999), 
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anchored  instruction  (Cognition  & Technology  Group  at  Vanderbilt,  1992),  cognitive  apprenticeships  (Collins, 
Brown,  & Newman,  1989),  reciprocal  teaching  (Palinscar  & Brown,  1984),  and  goal-based  scenarios  (Schank, 
Berman,  & Macpherson,  1999).  Although  somewhat  varied  in  their  scope,  and  methods,  these  approaches  share 
similar  key  epistemological  foundations  and  assumptions  about  the  nature  of  learning. 

In  these  constructivist  learning  environments,  the  roles  of  both  students  and  teachers  change  differently 
from  those  in  objectivist  learning  environments:  students  move  toward  more  self-reliance  and  peer  coaching  and 
teachers  function  more  as  facilitators  than  as  lecturers.  Constructivists  emphasize  the  process-oriented  learning 
environments  rather  than  content-oriented  learning  environments  (Bannan-Ritland,  Dabbagh,  & Murphy,  2000).  In 
the  process  oriented  learning  environments,  learners  are  usually  required  “to  examine  thinking  and  learning 
processes;  collect,  record  and  analyze  data;  formulate  and  test  hypothesis;  reflect  on  previous  understandings;  and 
construct  their  own  meaning”  (Crotty,  1994,  p.  31)  in  a variety  of  content  areas  (cited  in  Bannan-Ritland  et  al., 

2000) .  These  skills  are  closely  related  with  self-regulated  learning. 

However,  students,  especially  less  expert  learners,  are  likely  to  be  using  passive  learning  strategies  and  they 
are  not  likely  to  adopt  more  active  learning  strategies  (Land  & Hannafin,  2000).  Those  passive  strategies  are 
remarkably  persistent  and  enduring  because  students  may  have  a lifetime  of  experience  in  passive  learning 
environments.  These  learners  do  heavily  rely  on  knowledge  being  delivered  to  them  in  terms  of  lectures  and 
presentations  and  are  not  able  to  learn  from  new  situations  without  someone  there  to  package  the  new  knowledge  for 
them. 

More  expert  learners,  on  the  other  hand,  use  a variety  of  cognitive  strategies  and  self-regulation  procedures 
to  plan  and  pursue  goals,  integrate  new  knowledge  with  existing  knowledge,  formulate  questions  and  inferences,  and 
continually  review  and  reorganize  their  thinking  (Bereiter  & Scardamlia,  1989;  Scardamalia,  Bereiter,  McLean, 
Swallow,  & Woodruff,  1989).  Given  that  not  all  students  are  expert  learners,  it  is  important  to  assist  learners  with 
the  strategies  and  procedures  that  can  assist  with  learning  process.  Supporting  and  facilitating  self-regulated 
learning  may  lead  to  more  satisfactory  and  rewarding  learning  experiences. 

Higher  Education  and  Self- Regulated  Learning 

To  date,  research  and  development  efforts  related  to  self-regulated  learning  have  focused  primarily  on  K- 12 
students.  However  the  need  for  self -regulated  learning  with  adult  learners  has  gained  an  increased  focus.  College 
students  need  self-regulated  learning  because  they  have  more  freedom  than  secondary  students  (Paris  & Paris, 

2001) .  Research  has  indicated  that  new  students  in  particular  experience  difficulty  in  balancing  the  social  demands 
versus  the  educational  demands  of  the  higher  education  experience  (Zimmerman,  Greenberg,  & Weinstein,  1994). 
Burd  (1996)  estimated  that  one  third  of  the  students  who  enter  colleges  and  universities  will  be  under-prepared  or 
lack  the  skills  needed  to  be  successful  learners  (Ley  & Young,  1998).  “About  three  fourths  of  the  higher  education 
institutions  that  enrolled  freshmen  offered  at  least  one  remedial  reading,  writing,  or  mathematics  course  in  Fall 
1995”  (U.S.  Department  of  Education,  1996.  p.  1).  Ley  & Young  (1998)  argued  that  the  issue  is  not  a question  of 
whether  or  not  to  accept  under-prepared  college  applicants  any  more,  but  how  to  identify  those  students  and  assist 
them. 

An  easy  way  to  help  less  expert  learners  in  universities  is  to  offer  a how  to  learn  course  or  a learning 
strategies  course  (Hofer,  Yu,  & Pintrich,  1998).  Weinstein,  Husman,  and  Dierking  (2000)  suggested  that  an  adjunct 
course  approach  is  necessary,  especially  at  the  college  level,  where  it  is  very  unlikely  that  college  faculty  will  be 
able  to  teach  general  learning  and  self-regulatory  strategies  in  their  discipline-specific  courses.  This  implies  that  it  is 
very  hard  to  implement  the  integrated  course  at  the  college  level,  where  instruction  in  learning  strategies  is 
embedded  in  the  context  of  a regular  disciplinary  course  (e.g.,  chemistry,  history,  sociology).  Although  it  may  be 
easier  to  implement  an  adjunct  course  than  an  integrated  course  at  the  college  level,  one  of  the  main  drawbacks  to  an 
adjunct  course  relates  to  the  transfer  of  learning  (Hofer  et  al,  1998).  Therefore,  researchers  emphasize  the 
importance  of  applying  the  strategies  learned  in  the  adjunct  course  into  other  courses.  An  integrated  course  at  the 
college  level,  however,  still  faces  the  same  problem  of  transfer  strategies  learned  in  one  context  (e.g.,  a chemistry 
course)  and  to  other  disciplinary  courses  (Simpson,  Hynd,  Nist,  & Burrell,  1997).  How  to  support  the  less  expert 
learners  at  the  college  level  in  various  courses  still  needs  to  be  investigated. 

Another  population  that  could  benefit  from  self-regulated  learning  skills  are  graduate  students.  With  the 
needs  of  renewing  and  deepening  their  knowledge  and  skills,  many  adults  return  to  school  to  take  graduate  courses. 
However,  for  these  graduate  students,  adjunct  courses  may  not  be  an  appropriate  option.  One  possible  approach  to 
this  issue  is  to  provide  learners  with  cognitive  tools  that  are  designed  to  support  the  process  of  self-regulated 
learning  within  a specific  context.  A discussion  about  the  cognitive  tools  follows  in  the  next  section. 

Definition  of  Self-regulated  Learning 
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The  key  issue  defining  self-regulated  learning  is  not  whether  learning  occurs  as  a socially  isolated  event, 
but  rather  whether  the  learner  exhibits  personal  initiative,  perseverance,  and  adaptive  skill  in  pursuing  it 
(Zimmerman,  2001).  This  means  that  self-regulated  learning  does  not  have  to  be  limited  to  the  application  to  the 
solo  cognition  (Anderson,  Reder,  & Simon,  1996,  1997).  Self-regulated  learning  can  be  applied  to  the  learning 
based  on  the  situated  learning  theories  that  emphasize  the  social  nature  of  cognition  and  meaning  (Greeno,  1998; 
Resnick,  1987).  This  view  is  also  supported  by  the  Vygotskian  view  (McCaslin  & Hickey,  2001),  social  cognitive 
view  (Shunck,  2001),  and  constructivist  view  (Paris,  Byrnes,  & Paris,  2001)  of  self-regulated  learning. 

Self -regulated  learning  is  a form  of  learning  that  places  great  emphasis  on  learners’  active  role  and  control 
of  their  own  learning.  Self-regulated  learning  involves  the  conscious  awareness  and  modification  of  cognitive  as 
well  as  affective  processes  that  are  required  for  success  in  an  educational  environment  (Corno  and  Mandinach, 

1983).  It  can  be  contrasted  with  passive  learning,  a form  of  learning  whereby  learners  are  assumed  to  absorb 
information  presented  to  them  by  a lecturer  or  some  form  of  media  such  as  a film. 

Proponents  of  self-regulated  learning  espouse  that  it  is  more  effective  than  passive  learning.  They  also 
believe  that  it  is  more  important  because  learners  who  only  know  how  to  receive  information  passively  from  others 
won’t  be  successful  as  lifelong  learners  in  real  world  situations  where  the  knowledge  to  be  gained  is  not  neatly 
packaged  for  them. 

Although  each  theory  in  the  literature  about  self-regulated  learning  (e.g.,  operant,  phenomenological, 
information  processing,  social  cognitive,  volitional,  Vygotskian,  and  constructivist  views)  has  a different  view  to 
explain  learning  itself,  most  of  them  share  some  aspects  in  explaining  about  self-regulated  learning.  Pintrich  (2000) 
proposed  four  assumptions  about  self-regulated  learning  that  can  be  found  throughout  the  theories.  Pintrich’ s 
assumptions  on  self-regulated  learning  include:  (a)  that  is  an  active,  constructive  process,  (b)  that  learners  have 
potential  forcontrol,  (c)  that  learners  establish  goal,  criterion,  or  standard,  and  (d)  that  mediators  (e.g.,  teachers, 
computer  based  tools)  play  an  important  role.  From  these  common  assumptions,  Pintrich  (2000)  defined  self- 
regulated  learning  as  “an  active,  constructive  process  whereby  learners  set  goals  for  their  learning  and  then  attempt 
to  monitor,  regulate,  and  control  their  cognition,  motivation,  and  behavior,  guided  and  constrained  by  their  goals  and 
the  contextual  features  in  the  environment”  (p.  453).  This  definition  does  not  explicitly  tell  about  planning,  through 
which  learners  select  and  manipulate  the  strategies  necessary  to  achieve  the  goals  and  come  up  with  the  paths  to 
follow.  However,  various  models  of  self-regulated  learning  (e.g.,  Winne  & Stockley,  1998;  Zimmerman,  1998) 
have  planning  in  their  models  and  described  about  it  explicitly.  If  a planning  is  added  to  this  definition,  in  the 
simplest  sense,  self-regulated  learning  is  an  active,  constructive  process  whereby  learners  set  goals  for  their  learning, 
plan  how  to  achieve  the  goals,  and  then  attempt  to  monitor,  regulate,  and  control  their  cognition,  motivation,  and 
behavior,  guided  and  constrained  by  their  goals  and  the  contextual  features  in  the  environment. 

According  to  this  more  elaborated  definition,  a process  of  self-regulated  learning  can  be  described  as 
following.  Self-regulated  learners  know  when  to  engage  in  learning,  set  goals  for  learning,  plan  to  achieve  the  goal 
through  which  learners  select  the  strategies  to  use  and  decide  orders  to  follow.  As  they  move  toward  the  goals, 
learners  monitor  their  progress,  cognition  and  motivation.  If  they  find  their  progress  toward  goal  does  not  follow 
their  plans  or  encounter  obstacles,  learners  try  to  modify  the  plans  or  even  give  up  the  initial  goals,  to  manage 
motivation  and  adjust  their  strategies  for  making  progress. 

Computers  as  Cognitive  Tools 

Jonassen  and  Reeves  (1996)  offered  the  following  definition  of  cognitive  tools:  “Cognitive  tools  refer  to 
technologies,  tangible  or  intangible,  that  enhance  the  cognitive  powers  of  human  beings  during  thinking,  problem 
solving,  and  learning.  Written  language,  mathematical  notation,  and  most  recently,  the  universal  computer  are 
examples  of  cognitive  tools”  (p.  693).  One  of  the  rationales  for  using  cognitive  tools  is  that  a portion  of  the 
cognitive  power  used  by  an  individual  resides  in  other  people,  artifacts,  or  tools  created  by  the  larger  society  (Pea, 
1993;  Salomon,  1993a).  A lot  of  computer  cognitive  tools  have  been  developed  starting  with  the  Taylor’s  (1980) 
conception  of  computers  in  education  as  “Tutors,  Tools,  and  Tutees,”  to  more  sophisticated  cognitive  tools  as 
described  by  Lajoie  (2000). 

With  the  technological  developments,  such  as  multimedia  and  the  Internet,  various  Web-based  cognitive 
tools  have  been  developed.  Even  many  successful  non-Web  based  systems  are  going  “online,”  making  their  tools 
and  resources  accessible  via  the  Web  (Sugrue,  2000).  The  Web  has  great  potential  as  a cognitive  tool  with  its 
inherent  characteristics:  easy  access  to  the  vast  amount  of  information  distributed  throughout  the  Internet  and 
asynchronous  and  synchronous  communication  tools  that  can  enhance  communication  between  learners  and 
learners,  learners  and  teachers,  and  teachers  and  teachers. 

One  of  the  roles  that  can  be  supported  by  computers  as  cognitive  tools  is  similar  to  the  roles  played  by 
peers,  mentors,  experts  or  teachers  in  more  traditional  teaching-learning  environments.  For  example,  a cognitive 
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tool  can  be  developed  to  provide  guidance  and  hints  that  provide  just  enough  support  for  a learner  to  move  from  one 
stage  of  a learning  task  to  another.  With  this  point,  computers  as  cognitive  tools  can  support  self-regulated  learning. 
Schunk  and  Zimmerman  (1996)  emphasized  the  importance  of  the  social  source  to  support  self-regulated  learning. 
Learners  can  acquire  self-regulatory  knowledge  and  skill  from  this  social  source  through  modeling,  verbal 
instruction,  physical  guidance,  corrective  feedback,  social  structuring,  supervision  and  monitoring,  peer  teaching, 
cooperative  learning,  and  reciprocal  teaching.  Therefore,  how  computers  as  cognitive  tools  can  support  self- 
regulated  learning  is  worthwhile  to  be  investigated. 


Methodology 

This  study  focused  on  how  to  facilitate  self -regulated  learning  within  the  higher  education  by  using  the 
Web  technology  as  a cognitive  tool.  For  this  purpose,  this  study  examined  students’  use  of  the  SRLS  tool  in  a post 
secondary  course,  with  particular  interest  in  how  the  use  of  the  SRLS  tool  influence  their  self -regulated  learning,  and 
the  factors  that  influence  the  students’  use  of  the  SRLS  tool.  A pilot  study  was  conducted  to  determine  the 
feasibility  of  implementing  a SRLS  tool  within  the  context  of  a course  and  to  identify  any  problems.  Throughout 
data  collection  and  analysis,  this  study  was  guided  by  the  following  questions: 

1.  In  what  ways  does  the  SRLS  tool  affect  students’  self-regulated  learning? 

2.  What  factors  influence  the  students’  use  of  the  SRLS  tool? 

Design  of  the  SRLS  Tool 

Social  sources  are  essential  to  help  students  self-regulate  their  learning  (Schunk  & Zimmerman,  1996; 
McCaslin  & Hickey,  2001;  Schunk,  2001).  Social  source  includes  parents,  coaches,  teachers,  siblings,  friends,  and 
classmates.  Learners  can  acquire  self-regulatory  knowledge  and  skill  from  this  social  source  through  modeling, 
verbal  instruction,  physical  guidance,  corrective  feedback,  social  structuring,  supervision  and  monitoring,  peer 
teaching,  cooperative  learning,  and  reciprocal  teaching.  Learners  also  need  to  have  chances  to  rehearse  and  develop 
the  self-regulated  learning  on  their  own  (Schunk  & Zimmerman,  1996).  Learners  cannot  fully  self-regulate  their 
learning  unless  they  exercise  personal  choice  or  control.  How  social  support  and  self-directed  practice  are  arranged 
in  teaching  and  learning  environments  still  needs  to  be  investigated.  This  research  focuses  on  social  support  where 
the  students  are  given  the  chances  to  have  self-directed  experiences.  One  possible  approach  to  this  issue  is  to 
provide  learners  with  cognitive  tools  that  can  support  the  process  of  self-regulated  learning. 

Salomon  (1993b)  argued  that  the  computer  as  a cognitive  tool  can  become  a partner  with  the  novice  learner 
by  providing  guidance  or  a scaffold  as  the  novice  undertakes  a complex  task.  However,  it  should  be  students  not  the 
computers  to  set  the  learning  goals,  plan  to  achieve  the  goal,  monitor  their  progress  towards  the  goal,  and  reflect  the 
whole  process.  The  role  of  computers  should  be  to  promote  and  facilitate  students  to  maximize  use  of  their 
intelligence  and  knowledge.  And  the  design  of  the  tool  will  focus  on  both  “effect  of’  technology  and  “effect  with’’ 
technology  (Salomon,  1993a).  In  other  words,  this  tool  design  will  try  to  provide  quality  scaffolding  that  entails 
metacognitive  guidance  to  facilitate  students  learning  (the  “effect  of’  technology)  and  also  support  for  off-loading 
and  task  dividing  that  try  to  ease  students’  cognitive  burden  (the  “effect  with’’  technology). 

The  SRLS  tool  has  built-in  scaffolds  as  the  form  of  a template  with  guided  questions  to  help  students  with 
self-regulatory  activities.  Duffy  and  Cunningham  (1996)  defined  scaffolding  as  any  type  of  support  for  learning. 

This  includes  not  only  the  support  of  other  individuals  but  also  "any  artifacts  in  the  environment  that  afford  support” 
(p.  183). 

When  students  are  answering  the  guided  questions,  it  is  expected  that  they  would  engage  in  self-regulatory 
activities  comparable  to  metacognitive  processes  involving  goal  setting,  planning,  monitoring,  evaluating,  and 
reflecting.  These  activities  can  be  divided  into  two  phases.  The  first  phase  is  focused  on  goal  setting  and  planning. 
And  the  later  phase  is  focused  on  monitoring,  evaluating,  and  reflection. 

The  main  point  to  design  the  goal  setting  and  planning  phase  is  to  help  students  set  a learning  goal  that  is 
specific  enough  to  guide  their  learning  and  let  the  students  make  a plan  to  achieve  the  goal.  For  this  purpose,  a 
template  with  guided  questions  will  be  used.  The  first  question  asks  students  to  set  a short-term  and  specific 
learning  goal.  Although  the  question  cannot  guarantee  if  the  students  set  a goal  specific  enough  to  guide  their 
learning,  the  following  questions  will  play  a role  to  guide  the  students  to  specify  the  goal  to  guide  their  learning  and 
make  a plan  to  achieve  the  goal. 

The  template  with  guided  questions  provide  the  structure  and  play  a role  of  prompts  that  facilitate  students 
to  engage  in  the  targeted  activities  such  as  goal  setting,  and  planning  here.  The  template  supports  the  function  that 
Lajoie  (1993)  described  as  sharing  “the  cognitive  load  by  providing  support  for  lower  level  cognitive  skills  so  that 
(cognitive)  resources  are  left  over  for  higher  order  thinking  skills’’  (p.  261). 
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The  following  guided  questions  were  used  with  a template  for  goal  setting  and  planning. 

• What  are  my  new  specific  learning  goals? 

• What  is  the  target  date  for  the  goals? 

• What  are  my  actions  or  steps  that  I will  take  to  accomplish  these  goals? 

• How  will  I know  I have  accomplished  my  goals  by? 

• What  are  Possible  blocks,  both  personal  and  external,  that  may  interfere  with  accomplishing  these  goals 

and  how  can  I overcome  them  are? 

• Who  can  I go  to  for  help  with  my  goals? 

• How  confident  am  I that  I will  accomplish  these  goals? 

These  goal  setting  and  planning  activities  are  also  expected  to  improve  students  ’ self-efficacy.  Setting 
short-term  goals  involving  specific  performance  standards  is  likely  to  lead  to  successful  performance,  and  so  to 
enhance  self-efficacy  and  positive  self-reinforcement  because  short-term  goals  look  more  manageable  to  students 
and  specific  goals  provide  a clear  and  specific  guide  for  the  type  and  amount  of  effort  needed  to  accomplish  the 
goals  (Bandura,  1986;  Schunk,  1990).  Completing  a series  of  short-term  goals  often  requires  fewer  steps,  and 
results  in  fewer  errors,  than  trying  to  devise  and  implement  global  plan  for  reaching  the  long-term  goal. 

The  main  point  to  design  the  monitoring  and  reflection  phase  is  to  help  students  to  monitor  their  progress 
towards  the  goal  and  reflect  what  and  how  they  did  for  it.  For  this  purpose,  a template  with  several  guided  questions 
will  be  used  as  in  the  goal  setting  and  planning  phase.  These  questions  are  intended  to  prompt  students  to  engage  in 
metacognitive  activities  so  that  they  can  monitor  and  evaluate  their  learning  progress  and  have  an  opportunity  to 
reflect  what  they  did.  And  notification  messages  that  let  students  know  what  they  need  to  do  in  the  tool  when  they 
log  in  and  email  messages  will  be  sent  to  the  student  when  they  have  to  monitor  and  evaluate. 

Metacognitive  processes  can  be  supported  when  the  students  can  have  a chance  to  recap  their  problem- 
solving steps  by  reviewing  the  artifacts  they  created  as  a representation  of  their  internal  thoughts.  And  when  such 
physical  representation  provides  opportunities  for  the  learners  to  inspect  and  reflect  on  their  solution  strategies, 
generalized  metacognitive  awareness  is  prompted  (Lajoie,  1993).  The  SRLS  tool  can  help  students  to  be  involved  in 
these  processes  by  storing  their  artifacts  such  as  the  goals,  plans,  and  reflections  from  the  beginning. 

The  following  guided  questions  are  the  possible  questions  that  will  be  used  with  a template  for  monitoring 
and  evaluating.  These  will  also  be  modified  through  an  interview  with  a course  instructor  or  a pilot  test. 

• How  satisfied  am  I with  my  previous  goal  accomplishment? 

• What  were  the  reasons  for  accomplishing  or  not  accomplishing  my  goals? 

• What  is  your  general  reflection? 

Implementation  of  the  SRLS  Tool 

This  study  employed  a case  study  approach.  The  case  was  the  course  that  provided  the  introduction  to 
computer  to  pre-service  teachers  in  spring,  2002.  Forty  one  students  were  enrolled.  The  students  used  a SRLS  tool 
for  their  final  project  which  was  to  create  a Web-quest  for  their  future  students.  The  students  spent  approximately  3 
weeks  finishing  the  projects.  The  students  were  required  to  set  goals  for  the  projects  4 times  and  to  reflect  on  the 
goal  achievement  4 times  through  the  SRLS  tool. 

Data  Collection  and  Analysis 

At  the  end  of  the  course,  participant  opinions  regarding  the  tool  use  through  an  open-ended  questionnaire. 

It  asked  the  participants  such  things  as  what  they  did  and  did  not  like  about  the  elements  of  the  tool,  what  helped 
them,  what  did  not  help  them,  whether  they  felt  that  a SRLS  tool-related  activities  that  were  required  of  them  were 
appropriate  for  their  needs,  what  changes  they  would  make  if  they  could,  and  whether  they  felt  they  attained  the 
objectives.  Thirty  six  students  among  forty  one  students  volunteered  to  responded  to  the  questionnaire.  Constant 
comparative  method  of  analysis  (Strauss  & Corbin,  1998)  was  chosen  as  a method  of  analyzing  transcripts  of  the 
questionnaires  in  searching  for  common  themes  that  emerged  from  the  data. 

Findings 

Results  from  the  pilot  study  indicated  that  the  SRLS  tool  played  the  intended  role  for  supporting  the 
learning  despite  some  negative  opinions  from  some  of  the  students.  One  of  the  positive  remarks  about  the  tool  was 
that  it  helped  motivate  the  students  to  get  the  task  done.  The  tool  informed  them  of  their  accomplishment  through 
monitoring  and  evaluating  activities  and  it  helped  them  continue  to  complete  their  tasks.  This  finding  is  in  parallel 
with  the  literature  on  self-efficacy.  The  experience  of  successful  performance  enhances  self-efficacy  and  it  helps  the 
students  maintain  motivation  for  the  task  (Schunk  & Ertmer,  2000).  It  is  also  important  to  notice  that  the  short-term 
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goals  helped  students  because  completing  a series  of  short-term  goals  often  requires  fewer  steps,  and  results  in  fewer 
errors.  One  commented: 

It  helped]  divide  one  big  thing  into  small  parts.  It  also  informed  my  accomplishment,  thus 
motivate  me  to  get  the  task  done. 

Another  positive  remark  is  that  it  helped  time  management  of  the  students.  Students  indicated  that 
continuous  planning  and  monitoring  helped  keep  them  on  track.  It  helped  the  students  organize  thinking  into 
priorities  by  facilitating  the  students  to  divide  one  big  thing  (long-term  goal)  into  small  parts  (short-term  goals)  and 
to  order  them  to  achieve  the  big  thing.  The  tool  also  helped  the  students’  reflection  on  the  project  process.  Through 
the  reflection,  some  of  the  students  reported  they  tried  to  figure  out  why  they  did  not  do  good  jobs  this  time  and  how 
to  improve  next  time. 

Two  important  factors  that  are  regarded  to  affect  the  students’  perception  about  the  effectiveness  and 
efficiency  of  the  tool  are  the  characteristics  of  the  task  and  the  individuals’  motivation  level.  Some  of  the  students 
among  those  who  showed  the  tool  was  not  necessary  reported  that  the  project  was  too  easy  for  them  and  three  weeks 
were  short  enough  not  to  use  the  tool.  This  implies  that  using  the  tool  may  be  bothersome  to  the  students  if  the  task 
is  too  easy  to  them  or  needs  short  period  to  finish  it.  Another  negative  remark  was  from  those  who  felt  that  they 
were  well  motivated.  They  mentioned  that  they  were  motivated  enough  to  complete  the  projects  themselves  without 
using  the  tool. 

The  findings  from  this  pilot  study  influenced  the  sampling  for  the  case  in  the  future  study.  The  case  should 
have  a long-term  project  that  needs  enough  periods  to  use  the  SRLS  tool.  The  pilot  study  also  supports  the  necessity 
to  investigate  the  interaction  between  the  individual  difference,  the  tool,  and  other  factors  like  task  characteristics. 
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Abstract 

Teachers'  implementation  of  technology-enhanced  student -centered  learning  environments  will  be  affected 
by  their  beliefs  about  effective  practices,  particularly  their  beliefs  about  student -centered  learning.  In  order  for 
student-centered  programs  to  be  used  as  intended,  designers  must  be  aware  of  the  key  issues  that  will  shape  their 
implementation  and  the  beliefs  teachers  hold  about  these  issues.  This  case  study  examined  15  teachers’  beliefs 
about  student -centered  learning  as  they  implemented  Alien  Rescue,  a computer-based  program  for  middle  school 
science  that  was  designed  to  create  a student -centered  learning  environment  in  the  classroom.  Considerations  for  the 
design  of  similar  programs  are  offered. 

Student -Centered  Learning  Environments 

Concurrent  interest  in  learning  guided  by  a constructivist  perspective  and  advances  in  computer  technology 
have  led  to  a renewed  interest  in  student -centered  learning  (Land  & Hannifin,  2000).  Student-centered  learning 
environments  (SCLEs)  differ  from  teacher-directed  instructional  activities  in  their  assumptions  about  the  roles  of 
both  the  student  and  teacher  in  learning  (Hannifin,  Hill,  & Land,  1997).  Learning  in  SCLEs  is  driven  by  a problem 
or  project  (Jonassen,  2000)  in  which  the  learner  must  develop  a solution  or  product  that  meets  certain  specifications. 
Students  control  their  process  as  they  work  toward  this  goal,  determining  what  their  learning  needs  are  and  how  they 
will  meet  them.  Teachers  facilitate  student  work  rather  than  directing  it. 

Though  interest  in  student-centered  learning  spanned  much  of  the  twentieth  century,  it  largely  failed  to  take 
root  in  schools.  Cuban  (1983)  noted  “a  seemingly  stubborn  continuity  in  teacher-centered  instruction  despite  intense 
reform  efforts  to  move  classroom  practices  toward  instruction  that  was  more  learner  centered”  (p.  160).  Hannifin 
and  Land  (2000)  argue  that  the  impending  ubiquity  of  powerful  technologies  make  the  transition  to  student-centered 
learning  inevitable,  but  the  same  reasons  for  the  resistance  Cuban  noted  may  still  apply  and  in  fact  be  exacerbated 
by  technology.  Implementation  of  technology  enhanced  student-centered  programs  requires  that  teachers 
simultaneously  integrate  technology  and  embrace  pedagogical  approaches  that  may  be  unfamiliar  to  them.  Both 
research  done  on  the  barriers  to  technology  integration  (Ertmer,  1999)  and  teachers’  resistance  to  pedagogical 
change  (Richardson,  1990)  suggest  that  the  double-barreled  innovation  that  technology  enhanced  student-centered 
learning  environments  represent  may  prove  intimidating  for  teachers. 

Teachers'  Beliefs 

When  confronted  by  novel  situations  in  which  they  lack  knowledge  structures  and  cognitive  strategies, 
people  fall  back  on  their  beliefs  to  guide  the  decisions  they  make  (Pajares,  1992).  This  may  be  especially  true  in 
teaching,  which  Nespor  (1987)  describes  as  an  entangled  domain  because  of  the  numerous  situations  teachers 
encounter  which  have  overlapping  but  not  completely  analogous  characteristics  with  other  situations,  thereby 
requiring  teachers  to  frequently  make  decisions  in  the  absence  of  certainty  about  outcomes.  Eor  this  reason, 
researchers  have  argued  that  a greater  understanding  of  teachers’  beliefs  is  essential  to  the  improvement  of 
educational  practices  (Eang,  1996;  Lumpe,  Haney,  & Czerniak,  1998;  Tobin,  Tippins,  & Gallard,  1994). 

Teachers  implementing  a technology  enhanced  student-centered  program  are  likely  to  rely  on  the 
educational  beliefs  they  have  developed  during  their  experiences  both  as  a student  and  a teacher  to  navigate  the 
process.  These  beliefs  will  shape  the  way  in  which  a program  is  implemented,  and  ultimately  student  outcomes. 
Designers  who  assume  that  teachers  will  conform  to  guidelines  published  in  a teacher’s  manual  or  alter  their  practice 
to  fit  the  theoretical  underpinnings  that  guided  the  design  of  an  educational  program  may  find  themselves  seriously 
mistaken.  Eor  example,  Guskey  (1986)  found  evidence  that  in  the  absence  of  commitment  to  an  instructional 
innovation,  teachers  often  altered  the  practice  to  the  point  that  it  was  no  longer  effective.  Even  teachers  who  are 
aware  of  the  designers’  intentions  may  disregard  them  if  they  are  at  odds  with  their  own  beliefs.  Nespor  (1987) 
argued  that  beliefs  are  far  more  influential  than  their  knowledge  in  determining  how  people  define  problems  and  that 
they  are  much  better  predictors  of  their  behavior.  Therefore,  the  effective  design  of  student-centered  programs  will 
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need  to  take  into  account  teachers’  beliefs  about  student-centered  learning,  and  how  these  beliefs  are  likely  to  shape 
their  implementation  of  programs  that  are  designed  to  be  student-centered. 

The  purpose  of  this  study  was  to  identify  key  issues  in  the  implementation  of  a computer-based  program 
designed  to  support  student -centered  learning  and  to  examine  teachers’  beliefs  about  those  issues.  The  findings  can 
help  designers  recognize  some  of  the  factors  that  can  affect  how  teachers  implement  student-centered  programs, 
which  can  inform  both  the  design  of  these  programs  and  professional  development  workshops  that  focus  on  student- 
centered  practices.  This  study  can  also  bring  to  light  issues  to  be  considered  in  future  investigations  of  teachers’ 
beliefs  about  student -centered  practices,  how  these  beliefs  guide  the  implementation  of  student-centered  programs, 
and  in  turn,  how  implementation  variables  affect  student  performance  and  motivation. 

Method 

This  paper  reports  a case  study  of  teachers  who  implemented  Rescue,  a computer-based  program 

designed  to  support  student -centered  learning.  Fifteen  middle  school  science  teachers  participated.  The  number  of 
years  of  teaching  experience  of  these  teachers  ranged  from  one  to  twenty-five.  Most  of  the  teachers  who  participated 
in  this  study  had  students  who  received  special  services,  including  resource,  content  mastery  (CM),  and  classes  for 
the  emotionally  disturbed,  but  were  mainstreamed  for  science. 

These  teachers  used  Rescue  with  their  classes.  This  computer-based  program  presents  students  with  a 
complex  problem  to  solve.  In  order  to  save  the  lives  of  six  species  of  aliens,  students,  working  as  scientists  aboard  a 
space  station,  must  find  new  homes  in  our  solar  system  for  each  species.  All  resources  that  students  need  to  develop 
a solution  to  the  problem  are  contained  within  the  program.  These  include  informational  resources  on  the  aliens,  the 
planets  and  large  moons  of  our  solar  system,  and  related  science  concepts;  and  tools  students  can  use  to  acquire  and 
organize  information,  including  a simulation  in  which  students  can  design  and  launch  probes  to  gather  data  about 
our  solar  system  (Pedersen,  Liu,  & Williams,  in  press).  Alien  Rescue  is  aligned  with  the  national  standards  for 
science  education  and  requires  approximately  fifteen  45-minute  periods  to  complete.  Previous  research  suggests 
that  students  express  greater  intrinsic  motivation  for  their  work  within  AZ/en  Rescue  than  they  do  for  their  normal 
class  activities  (Pedersen,  in  press). 

The  teacher’s  manual  for  AZZen  Rescue  offers  guidelines  for  teachers  on  how  to  implement  the  program  so 
as  to  encourage  student  ownership  over  their  problem-solving  process.  An  underlying  assumption  of  our  design  of 
Alien  Rescue  was  that  the  role  of  the  teacher  is  critical  to  students’  success  and  learning.  Teachers  are  expected  to 
promote  the  social  construction  of  knowledge  by  encouraging  students  to  collaborate  with  their  peers,  help  students 
to  connect  science  concepts  m Alien  Rescue  to  their  prior  knowledge  and  other  areas  of  the  curriculum,  and  probe 
individual  students’  thinking  as  they  are  engaged  in  the  problem  so  as  to  promote  reflection  and  identify 
misconceptions.  The  teacher’s  manual  contains  extremely  detailed  suggestions  on  how  to  accomplish  these  tasks 
over  the  course  of  the  program.  However,  AZZe«  Rescue  is  flexible  enough  for  teachers  to  adapt  it  to  the  particular 
needs  of  their  students  and  curriculum,  and  this  very  flexibility  makes  it  possible  for  teachers  to  disregard 
suggestions  made  in  the  teacher’s  manual.  Implementation  of  Alien  Rescue  is,  therefore,  quite  dependent  upon  the 
decisions  of  the  classroom  teacher. 

Data  sources  included  interviews,  observations,  the  first  author’s  reflexive  journal,  and  focus  groups.  Nine 
teachers  participated  in  two  interviews  each  - one  before  they  used  Alien  Rescue  with  their  students  and  once  after 
their  classes  completed  the  program.  Interviews  ranged  from  25  to  90  minutes,  with  an  average  duration  of  55 
minutes.  Approximately  twenty-five  hours  of  observations  were  conducted  in  seven  teachers’  classes.  These 
observations  were  used  as  a springboard  to  question  teachers  as  to  their  reasons  for  imple  menting  the  program  as 
they  did.  Two  focus  groups  of  teachers  each  met  for  approximately  75  minutes  to  discuss  a preliminary  version  of 
the  findings  of  this  study.  These  focus  groups  provided  a vehicle  for  member  checking  and  an  opportunity  for 
teachers  to  react  to  and  expand  upon  each  other’s  points. 

The  study  was  conducted  over  the  course  of  one  academic  year  with  data  collection  and  analysis 
overlapping  for  much  of  that  time.  Data  analysis  was  consistent  with  the  constant  comparative  method  (Lincoln  & 
Cuba,  1985). 

Results 

Teachers’  Definitions  of  Student -Centered  Learning 

A difficulty  we  anticipated  in  examining  teachers’  beliefs  about  student-centered  learning  was  the  lack  of  a 
common  definition  for  this  approach.  Our  interviews  with  teachers  showed  that  our  concern  was  valid.  As  shown  in 
Table  1,  four  different  definitions  of  student -centered  learning  emerged  from  the  data. 
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Table  1:  Teachers’  Definitions  of  Student-Centered  Learning 


In  Student-Centered  Learning. . . 

Definition  1 

The  teacher  considers  the  interests  and  needs  of  the  students  in  the  class,  and  then 
provides  instruction  based  on  them.  The  teacher  tries  to  make  sure  that  students  acquire 
the  information  and  understand  the  concepts  presented  before  moving  on  to  more 
difficult  material.  The  teacher  takes  into  account  individual  differences  and  makes 
adjustments  to  accommodate  individual  students. 

Definition  2 

The  teacher  prepares  an  activity  that  requires  students  to  be  actively  engaged.  These 
activities  are  often  “hands-on”  and  collaborative,  but  they  do  not  need  to  be.  The  teacher 
explains  the  steps  students  need  to  go  through  in  the  activity,  and  helps  to  redirect 
students  if  they  have  trouble  following  the  steps. 

Definition  3 

The  teacher  presents  students  with  a complicated  activity  but  does  not  tell  students  how 
to  complete  it.  Students  must  figure  out  what  to  do,  which  means  that  they  sometimes  try 
things  that  don’t  work.  The  teacher  questions  students  about  their  thinking,  but  do  not 
solve  their  problems  for  them  or  tell  them  what  to  do.  When  students  encounter 
difficulties,  they  turn  to  their  peers  for  support;  therefore  collaboration  grows  naturally 
out  of  student -centered  learning. 

Definition  4 

The  teacher  presents  a topic  students  are  supposed  to  learn  about,  then  allows  each 
student  to  investigate  whatever  aspect  of  that  topic  interests  him  or  her.  This  means  that 
students  are  often  working  on  widely  different  projects  that  they  themselves  have 
developed.  If  students  have  difficulty  choosing  what  to  investigate  or  finding  materials, 
the  teacher  helps  them  by  asking  questions,  but  does  not  tell  them  what  to  do  or  provide  a 
model  or  detailed  expectations  for  a product.  The  teacher  questions  students  about  their 
work  and  students  present  what  they  learn  to  their  classmates. 

Definition  1 is  evocative  of  child-centered  learning,  in  which  teachers  and  schools  are  charged  with 
providing  developmentally  appropriate  educational  experiences  in  a safe  environment  so  as  to  promote  the  overall 
well-being  of  all  students  (Manning,  2000).  This  definition  was  evident  in  comments  such  as  this: 

I try  to  think  of  my  planning  as  student -centered.  You  know,  whenever  I write  a lesson  plan,  I hope 
that  it’s  student -centered.  I mean,  I make  it  so  that  the  objective  is  that  the  student  will  be  able  to. . .etc.,  you 

know.  So  I don’t  know  what  you  mean  by  that  because  I think  everything  I do  is  student-centered. 

Definition  2 suggests  that  some  teachers  view  any  activity  in  which  students  are  active  and  work  with  their 
classmates  as  student -centered.  Several  teachers  described  the  science  “labs”  (activities)  they  teach  as  student- 
centered  because  students  use  a variety  of  materials,  work  in  groups,  and  write  their  own  conclusions.  However,  the 
teacher  typically  decides  which  labs  to  use,  provides  students  with  a step-by-step  process  for  the  lab,  structures 
students’  time  and  format  for  their  conclusions,  then  reviews  students’  conclusions  in  class  to  make  sure  they 
reached  the  “right”  ones.  While  students  are  actively  engaged,  most  of  the  labs  teachers  described  were  teacher- 
directed  and  allowed  students  little  control  over  their  activity,  process,  or  outcome. 

Most  teachers  held  either  definition  3 or  4,  and  in  a focus  group,  teachers  agreed  that  student-centered 
learning  was  broad  enough  to  encomp  ass  both  definitions.  It  was  useful  to  separate  these  two  definitions  however 
because  two  teachers  who  originally  held  definition  4 had  a negative  opinion  of  student -centered  learning.  They 
believed  that  the  lack  of  an  established  goal  implied  in  definit  ion  4 made  SCL  too  difficult  and  potentially 
frustrating  for  students  this  age: 

I think  student-centered  learning. ..it  would  have  to  come  from  the  students.  In  sixth  grade,  they  don’t  know  enough 
for  it  to. . .where  to  go  look  for  something.  Student-centered  would  be  more  for  a child  that's  ready  to  take  off  on  his 
own  and  go  investigate  everything.  They're  so  basic  in  sixth  grade.  I mean,  they're  Just  learning  science  and  a lot  of 
the  students  never  had  such  an  intensive  program  as  they're  having  now.  And  so,  they  want  to  learn,  but  it's  a while 
before  they  can  go  off  on  their  own.  I do  believe  there  is  a place  for  student -centered  learning  within  our 
curriculum,  but  they're  not  ready. 

These  teachers  both  thought  A/ien  Rescue  was  effective  in  large  part  because  it  provided  a clear  goal,  which 
helped  to  focus  students’  work.  Their  experience  with  A//e«  Rescue  seemed  to  have  broadened  these  teachers’ 
definitions  of  SCL  to  include  activities  that,  like  Alien  Rescue,  have  established  goals. 

Because  of  the  varied  definitions  teachers  held  of  student -centered  learning,  after  the  first  round  of 
interviews,  we  did  not  use  this  term  in  the  abstract.  Instead,  we  discussed  “programs  like  AZ/e«  Rescue”  and 
“student -centered  learning  environments  \ikt  Alien  Rescue.”  We  did  this  so  as  to  ground  the  discussion  in  one 
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common  experience  that  teachers  could  reflect  upon,  and  so  that  we  could  examine  and  compare  the  role  of  their 
beliefs  in  this  one  type  of  environment. 

The  Teacher's  Role 

Most  teachers  used  the  term  “facilitator”  to  describe  their  role  both  during  their  regular  classes  and  during 
Alien  Rescue.  However,  the  activities  teachers  described  doing  in  this  role  differed  somewhat.  Table  2 provides  a 
compilation  of  the  tasks  that  teachers  said  they  normally  perform  in  each  of  the  two  settings. 


Table  2:  Teachers  ’ activities  in  different  settings 


Normal  class  activities 

Alien  Rescue 

Planning 

Plan  lessons  and  establish  procedures  for 
carrying  out  tasks  during  those  lessons 
Establish  objectives 
Make  sure  supplies  are  available 

Establish  checkpoints  or  deadlines  by 
which  students  must  complete  certain 
steps 

Instruction 

Get  students  excited  about  science  topics 
Demonstrate  processes 
Lead  discussions 

Relate  new  knowledge  to  students’  prior 
knowledge 

Ask  questions  about  science  concepts  to 
find  out  how  much  students  have 
learned,  then  provide  mini-lectures  on 
science  concepts  to  enhance  students’ 
knowledge 

Lead  discussions  to  get  students  to  share 
their  findings 

Motivating 

Students 

Praise  students 

Grade  students  to  motivate  them  and  hold 
them  accountable  for  homework 

Praise  students 

Behavior 

Redirect  students  who  get  off  task 
Prevent  children  from  being  mean  to  each 
other 

Prevent  students  who  misbehave  from 
disturbing  their  classmates 

Redirect  students  who  get  off  task 

During 

Answer  students’  questions 

Interact  with  students  one-on-one  to  ask 

activities 

Make  sure  students  are  not  struggling  and 
provide  help  to  those  who  are 
Make  sure  students  practice  safety 
procedures 

questions  about  their  science 
knowledge 

Interact  with  students  one-on-one  to  ask 
questions  about  their  process 
Encourage  students  to  work  together 
Check  students’  work  in  order  to  make 
sure  they  are  progressing  at  an 
appropriate  pace 

Give  feedback  on  students’  products  in 
time  for  them  to  make  revisbns 

Though  teachers  differed  in  the  tasks  they  identified  and  in  some  cases  disagreed  about  whether  certain 
tasks  were  important,  the  lists  in  Table  2 suggest  that  teachers  identify  more  tasks  in  which  they  have  control  for 
their  regular  class  activities  than  for  environments  like  Alien  Rescue.  For  example,  for  normal  class  activities, 
several  teachers  said  that  answering  students’  questions  and  helping  students  who  were  struggling  were  important 
tasks  for  the  teacher.  For  Alien  Rescue,  teachers  talked  more  about  asking  questions  to  gauge  student  thinking  and 
encourage  reflection.  As  another  example,  though  all  the  teachers  graded  students  during  A/ien  Rescue  (see  section 
on  Grading  below),  none  of  them  listed  this  as  an  important  aspect  of  their  role  during  the  program,  while  several 
teachers  identified  this  as  an  important  task  during  their  regular  classes.  While  the  data  is  too  open  ended  to  draw 
any  strong  conclusions,  it  does  raise  the  question  that,  though  teachers  frequently  refer  to  their  role  as  that  of  a 
facilitator,  that  what  they  actually  do  fluctuates  based  on  the  activity  in  which  their  students  are  engaged,  and  that  in 
many  cases  their  “facilitation”  actually  involves  providing  a great  deal  of  direction  and  structure. 

During  a focus  group,  teachers  were  given  the  compilation  of  tasks  teachers  said  they  performed  during 
Alien  Rescue  (as  shown  in  the  right  column  of  Table  2)  and  asked  to  rank  them  in  order  of  their  importance.  Two 
patterns  were  seen  in  their  responses.  First,  some  teachers  ranked  establishing  checkpoints  for  students  and 
checking  students’  work  to  make  sure  they  are  progressing  as  the  most  important  tasks  for  the  teacher  during 
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programs  like  Alien  Rescue.  A second  set  of  teachers  saw  interacting  with  students  one-on-one  to  question  their 
process  and  knowledge  as  the  most  important  tasks  for  the  teacher.  In  other  words,  some  teachers  believed  that 
structuring  the  experience  was  the  most  important  responsibility  of  the  teacher,  while  other  teachers  believed 
supporting  higher  order  thinking  was  the  most  important.  The  limited  number  of  participants  in  this  study  makes  it 
difficult  to  establish  clear  patterns,  but  this  contrast  merits  further  investigation. 

Collaboration 

Teachers  generally  believed  that  collaboration  is  a valuable  component  of  any  educational  approach,  not 
just  student-centered  learning.  Interestingly,  most  teachers  said  that  the  reason  for  having  students  collaborate  was 
so  that  they  would  develop  skills  in  working  together  that  they  would  need  throughout  their  lives  and  believed  that 
complex  activities  like  Alien  Rescue  could  promote  the  development  of  collaborative  skills.  In  other  words,  they 
saw  collaboration  as  an  end  in  itself:  students  should  collaborate  so  that  they  learn  how  to  collaborate  better.  Only 
two  teachers  expressed  additional  reasons  for  collaboration;  these  were  better  problem-solving  through  social 
negotiation  and  enhanced  communication. 

Grading 

Almost  all  the  teachers  expressed  the  belief  that  it  is  essential  to  grade  students  during  student -centered 
learning  activities  like  Alien  Rescue.  They  offered  two  reasons  for  this.  First,  most  teachers  argued  that  grades  are 
necessary  to  motivate  students  to  do  their  best  work,  and  that  this  incentive  in  turn  helps  students  to  learn  more. 
Several  teachers  said  that  they  believed  grades  are  only  necessary  to  motivate  students  who  are  not  self- motivated, 
but  because  the  teacher  must  be  consistent  and  fair,  all  students  need  to  be  graded.  Second,  several  teachers  reported 
that  parents  and  administrators  expect  grades  and  that  teachers  have  to  justify  the  grades  they  give  on  report  cards  by 
being  able  to  show  grades  on  tasks  students  have  performed  throughout  the  marking  period. 

Contrary  to  research  findings  that  suggest  that  grading  can  undermine  students’  intrinsic  motivation  (Butler 
& Nisan,  1986;  Harter,  1978),  only  one  of  the  teachers  interviewed  believed  that  grading  during  student -centered 
learning  could  be  detrimental.  Teachers  argued  either  that  most  students  did  not  mind  grades  or  that  they  in  fact 
wanted  to  be  graded.  The  latter,  several  teachers  argued,  was  especially  true  for  higher  achieving  students. 

Standardized  Tests 

Some  teachers  argued  that  student -centered  activities  like  Rescue  are  not  necessarily  helpful  in 

preparing  students  for  standardized  tests.  Teachers  were  concerned  that  these  tests  ask  specific  factual  knowledge 
questions  about  a wide  range  of  topics.  This  presents  two  problems.  First,  teachers  believed  that  student-centered 
programs  like  Alien  Rescue  were  effective  in  helping  students  to  develop  problem-solving  skills,  but  that  they  would 
not  necessarily  be  effective  at  helping  students  to  learn  factual  knowledge.  In  fact,  one  suggestion  made  in  a focus 
group  was  that  a component  be  added  lo  Alien  Rescue  that  tests  students’  acquisition  of  factual  knowledge,  such  as  a 
game  with  multiple -choice  questions  so  that  it  reflects  the  type  of  questions  students  are  likely  to  encounter  on 
standardized  tests.  Second,  most  teachers  believed  that  for  the  amount  of  concept  learning  that  occurs,  student- 
centered  activities  are  more  time-consuming  than  teacher-directed  ones.  They  would  therefore  use  only  a limited 
number  of  student-centered  activities  in  a year,  and  be  less  likely  to  use  these  activities  during  periods  when  they  are 
preparing  students  for  a standardized  test.  However,  in  a focus  group  teachers  discussed  potential  changes  to  the 
standardized  tests  their  students  would  take  in  the  near  future.  They  had  been  informed  that  the  new  tests  would 
focus  less  on  factual  knowledge  and  more  on  problem-solving.  However,  they  remained  cautious  and  said  this 
change  may  make  student-centered  activities  more  viable. 

Students’  Motivation 

The  teachers  who  participated  in  this  study  generally  believed  that  student-centered  activities  are  likely  to 
support  intrinsically  motivated  behavior  on  the  part  of  students.  As  evidence  of  this,  teachers  reported  that  during 
Alien  Rescue,  students  stayed  engaged  throughout  the  three-week  program,  were  rarely  off  task,  and  that 
absenteeism  and  behavior  problems  were  down.  Several  teachers  also  noted  that  students  who  are  typically 
unmotivated  during  science  were  more  motivated  than  usual.  One  teacher  commented,  “If  you  have  a good  student- 
centered  learning  project,  you’re  not  going  to  have  any  problem  with  motivation.’’ 

Though  teachers  agreed  that  Alien  Rescue  supports  students  in  adopting  an  intrinsic  motivational 
orientation,  they  disagreed  about  the  reasons.  Several  teachers  attributed  students’  motivation  to  qualities  of 
student-centered  learning.  For  example,  teachers  believed  that  the  active,  hands-on  nature  of  the  program  was  key 
to  students’  engagement,  and  that  establishing  a goal  for  them  to  work  toward  helped  to  motivate  them.  Others 
believed  that  students  were  motivated  by  the  fact  that  they  had  control  over  their  process,  though  in  a focus  group 
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was  split  over  this  topic  with  some  teachers  arguing  that  student  control  over  their  process  was  not  an  essential 
ingredient  of  intrinsic  motivation. 

In  contrast,  several  other  causes  to  which  teachers  attributed  students’  motivation  had  little  to  do  with  the 
qualities  of  student-centered  learning.  When  a focus  group  was  asked  to  rank  order  the  factors  that  they  felt  most 
contributed  to  students’  motivation,  teachers  overwhelmingly  ranked  the  reasons  given  above  as  less  important  than 
another  factor:  the  computer-based  delivery  medium.  Almost  every  teacher  commented  that  students  were  motivated 
by  the  opportunity  to  use  computers  for  an  extended  period  of  time.  Several  teachers  said  that  the  rich  media  and 
science  fiction  premise  made  students  feel  as  though  they  were  playing  a game  or  solving  a puzzle  rather  than 
working  and  learning.  These  comments  suggest  that  teachers  believe  that  the  delivery  medium  and  the  sheer  novelty 
of  the  experience  may  account  in  large  part  for  students’  enhanced  interest  and  intrinsic  motivation. 

Students  with  Special  Needs 

One  of  the  concerns  mentioned  by  most  teachers  pertained  to  the  appropriateness  of  programs  like  Alien 
Rescue  for  some  students  with  special  needs.  Teachers  described  two  problems  that  can  make  student-centered 
programs  like  Alien  Rescue  problematic  for  many  of  these  students.  First,  for  students  who  read  below  grade  level, 
it  is  difficult  for  them  to  use  rich  text  resources  independently.  They  are  unable  to  find  the  information  they  need 
without  support.  Second,  a few  teachers  believed  that  some  special  needs  students  do  not  want  to  take  control  over 
their  learning  or  make  decisions  about  their  process.  As  a result,  they  will  not  develop  ownership  of  the  problem  or 
task  that  is  considered  essential  in  student-centered  learning.  They  want  teacher  direction,  and  when  teachers  do  not 
tell  them  what  to  do  they  simply  remain  passive  or  aimlessly  look  through  program  resources. 

Other  teachers  argued  that,  despite  these  difficulties,  student-centered  programs  are  appropriate  for  students 
with  special  needs.  Several  teachers  said  that  these  students  had  been  more  motivated  during  Ah'en  Rescue  than 
during  their  normal  science  classes.  They  believed  that  these  students  may  not  achieve  as  great  an  understanding  as 
other  students,  but  that  they  get  more  out  of  these  activities  than  they  do  out  of  activities  that  are  more  teacher- 
directed. 

Most  teachers  believed  that,  with  extra  support,  inclusion  students  could  participate  in  student-centered 
activities.  Teachers  primarily  offered  supports  that  could  be  provided  within  the  classroom,  such  as  greater  direction 
from  the  teacher,  one-on-one  or  small  group  help  from  a special  education  teacher,  and  pairing  inclusion  students 
with  regular  education  or  gifted  peers.  Some  teachers  felt  that  the  latter  works  well,  though  one  teacher  pointed  out 
that  this  could  lead  to  some  resentment  when  the  peer  feels  that  the  inclusion  student  is  not  doing  his  or  her  share. 
Teachers  did  suggest  one  modification  to  Alien  Rescue  might  be  useful  in  similar  programs.  They  suggested  that 
audio  recordings  of  text  passages  be  provided  for  those  students  who  have  difficulty  reading  independently,  and  that 
the  teacher  be  able  to  turn  this  option  off  for  students  who  should  read  without  assistance. 

Floundering 

Programs  like  Alien  Rescue  present  students  with  difficult  problems  that  take  a long  period  of  time  to  solve. 
Because  the  problem  is  complex  and  so  much  information  is  available,  some  students  flounder  at  various  points  in 
the  program,  either  pursuing  an  unproductive  plan  of  action  or  failing  to  develop  a plan  of  action  at  all.  Most  of  the 
teachers  who  participated  in  this  study  believed  that  this  is  not  necessarily  a problem,  and  that  the  difficulties 
students  encounter  eventually  lead  them  to  rethink  their  plan  of  action.  Some  teachers  pointed  out  that  floundering 
may  even  benefit  students  in  three  ways.  First,  students  are  later  able  to  explain  why  they  had  difficulties,  and  this 
reflection  helps  them  to  become  better  problem  solvers.  Second,  students  learn  to  deal  with  some  of  their 
frustrations  and  eventually  become  more  self-reliant  and  capable  of  handling  other  student-centered  activities. 

Finally,  initial  missteps  give  students  a better  understanding  of  the  nature  of  scientific  inquiry.  These  teachers 
generally  believed  that  they  could  help  students  who  flounder  by  questioning  them  and  getting  them  to  explain  what 
they  are  doing  rather  than  directing  the  students  to  work  in  particular  ways. 

However,  not  all  teachers  expressed  this  level  of  tolerance  for  students’  floundering.  A few  feared  that 
when  students  flounder,  they  waste  time  or  can  become  frustrated,  and  these  teachers  felt  responsible  for  fixing 
problems  that  block  student  progress. 

Community  Reaction 

The  teachers  who  participated  in  this  study  believed  that  the  adults  with  whom  they  work  - administrators, 
parents,  and  other  teachers  - would  be  mostly  supportive  of  their  use  of  student-centered  learning  activities.  In 
particular,  they  believed  that  most  administrators  are  quite  supportive  of  student-centered  learning,  and  in  some 
cases  administrators  even  actively  encourage  student-centered  approaches.  They  argued  that  administrators  no 
longer  expect  to  see  students  in  their  seats,  and  prefer  to  see  students  talking  to  each  other  and  doing  hands-on 
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activities.  However,  administrators  are  concerned  about  test  scores,  so  teachers  have  to  be  able  to  convince  them 
that  whatever  approach  they  use  will  be  effective  at  helping  students  do  well  on  standardized  tests. 

Teachers  also  believed  that  most  parents  would  be  supportive  of  their  decision  to  use  student-centered 
learning  activities.  However,  two  teachers  did  express  a concern  that  some  parents  may  complain  that  their  children 
are  just  playing  rather  than  actually  learning  science,  particularly  if  the  activity  were  computer-based.  This  concern 
could  actually  discourage  some  teachers  from  using  student -centered  activities. 

Discussion 

Teachers  are  more  likely  to  use  computer-based  student -centered  programs  in  ways  consistent  with  the 
designers’  intentions  if  these  programs  are  designed  with  their  beliefs  in  mind.  The  issues  raised  by  teachers 
participating  in  this  study  provide  the  grounds  for  some  suggestions  for  designers  to  consider  as  they  create  these 
programs  and  for  professional  development  workshops  focusing  on  student-centered  learning. 

Considerations  for  Designers 

Computer-based  programs  designed  to  support  student-centered  learning  in  the  classroom  are  still  relatively 
rare,  and  the  many  different  forms  these  programs  can  potentially  take  makes  generalizing  from  our  work  on  Alien 
Rescue  to  the  design  of  other  programs  a task  for  a high  degree  of  caution.  The  beliefs  teachers  expressed  in  this 
study  have  raised  a variety  of  design  considerations,  but  like  any  qualitative  study,  we  leave  it  to  the  readers  to 
determine  the  transferability  of  our  findings  (Lincoln  & Cuba,  1985)  to  their  work.  Given  this,  we  would  like  to 
offer  three  considerations  for  designers  that  we  believe  are  pertinent  to  student-centered  programs. 

Provide  scaffolds  for  students  with  special  needs.  Though  teachers  are  accustomed  to  modifying  materials 
and  providing  extra  support  for  students  with  special  needs,  computers  are  capable  of  sharing  this  responsibility. 

The  teachers  in  this  study  recommended  one  modification,  the  inclusion  of  audio  recordings  of  text  passages. 
However,  designers  of  computer-based  programs  should  explore  other  scaffolds  as  well.  Some  scaffolds  may  be 
unnecessary  for  many  students,  and  if  given  access  to  them,  these  scaffolds  may  oversimplify  students’  work. 
Therefore,  designers  may  want  to  consider  allowing  teachers  the  opportunity  to  select  which  scaffolds  will  be  active 
for  each  student. 

Support  factual  knowledge  acquisition.  One  concern  teachers  raised  in  this  study  was  that  student -centered 
programs  may  not  support  students’  acquisition  of  the  body  of  factual  knowledge  that  up  until  now  has  been  the 
focus  of  standardized  tests.  This  concern  has  been  raised  previously  in  the  literature  on  student -centered  approaches 
(Williams,  1993),  and  teachers’  comments  about  it  here  suggest  that  it  is  an  issue  that  must  be  addressed  by 
designers.  Even  if  these  tests  change,  teachers  are  likely  to  still  judge  the  effectiveness  of  instructional  materials 
based  at  least  in  part  on  how  much  factual  information  students  acquire,  even  though  this  runs  counter  to 
constructivist  views  of  learning,  where  learners  are  expected  to  acquire  facts  only  as  they  become  useful  to  the  task 
at  hand.  Still,  there  is  no  reason  why  student-centered  learning  should  not  lead  to  factual  knowledge  acquisition. 
SCLEs  are  typically  rich  in  factual  information  and  the  task  presented  in  these  environments  can  be  designed  to 
require  the  acquisition  and  application  of  many  facts.  However,  the  facts  students  learn  in  an  SCLE  may  not  be  easy 
to  assess  through  standardized  tests.  Students  may  not  all  acquire  the  same  specific  facts  within  an  SCLE  and  they 
may  be  unable  to  apply  their  newly  acquired  knowledge  to  respond  correctly  to  objective  questions  as  used  in 
standardized  tests.  Designers  of  student-centered  programs  should  consider  identifying  a core  set  of  “facts”  that 
students  should  learn  while  engaged  in  the  program,  and  provide  support  for  their  acquisition.  How  this  is  to  be 
done  without  engaging  students  in  activities  that  are  not  meaningful  within  the  SCLE  is  problematic.  Eor  example, 
the  suggestion  that  teachers  offered  in  this  study,  that  multiple -choice  style  questions  be  included  in  the  program,  is 
one  option,  but  it  is  an  awkward  one.  Such  questions  typically  detach  facts  from  contexts  in  which  they  are 
meaningful,  which  is  at  odds  with  the  theoretical  underpinnings  of  constructivism  and  student-centered  learning. 

Still,  this  issue  proved  to  be  important  enough  in  this  study  to  affect  several  teachers’  satisfaction  with  the  program, 
so  designers  need  to  consider  possible  methods  for  addressing  it. 

Capitalize  on  the  multimedia  affordances  of  computer  technology  to  create  new  learning  experiences  for 
students.  Alien  Rescue  is  rich  in  its  use  of  media.  The  program  incorporates  text,  graphics,  audio,  video,  3D  images, 
and  animation  throughout,  which  enhances  the  presentation  of  the  program  content.  A consensus  from  the  teachers 
in  this  study  is  that  the  rich  media  and  the  novel  experience  as  created  by  hypermedia  technology  is  an  important 
factor  for  students’  enhanced  interest  and  intrinsic  motivation  in  using  Alien  Rescue.  Though  the  development  of 
media-rich  programs  is  expensive,  the  cost  may  be  justified  by  its  impact  on  students’  motivation.  Also,  for  the  near 
future,  this  may  mean  that  CD  ROMs  offer  a better  delivery  option  than  the  Web  simply  because  of  issues 
associated  with  bandwidth  and  download  time  for  rich  media. 
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Suggestions  for  Professional  Development 

Our  experience  with  Alien  Rescue  has  suggested  a need  for  professional  development  opportunities  for 
teachers  prior  to  their  use  of  computer-based  programs  designed  to  support  student-centered  learning.  Our 
interactions  with  teachers  during  this  study  offer  a number  of  insights  for  the  design  of  professional  development 
workshops: 

Avoid  the  use  of  the  terms  “student-centered  learning”  and  “facilitator,  ” or  carefully  build  a common 
definition  with  teachers.  Our  findings  suggest  that  teachers  do  not  all  define  these  terms  in  the  same  way.  Their 
different  definitions  may  lead  to  miscommunication  and  discrepancies  between  the  teacher’s  implementation  of 
student-centered  programs  and  designers’  intentions.  A better  approach  would  be  to  anchor  discussions  of  pedagogy 
in  specific  examples  of  facilitation  strategies  useful  within  a given  program,  tying  these  strategies  to  a theory  of  how 
learning  occurs  within  the  environment. 

Address  the  many  benefits  of  collaboration.  The  teachers  who  participated  in  this  study  obviously  valued 
collaboration,  but  primarily  as  an  end  in  itself,  that  is,  so  that  students  develop  skills  for  working  with  others. 
Professional  development  workshops  can  help  teachers  to  recognize  benefits  that  impact  learning,  such  as  students’ 
development  of  an  understanding  of  the  role  of  collaboration  in  scientific  inquiry,  the  opportunity  to  identify 
misconceptions  during  collaborative  exchanges,  or  the  value  of  peer  modeling  for  students  on  both  ends  of  the 
dialog. 

Future  Research 

Finally,  a number  of  the  beliefs  teachers  expressed  in  this  study  deserve  further  examination  before 
suggestions  about  them  can  be  offered. 

Assessment  practices.  Teachers’  assumption  that  grading  during  student-centered  learning  activities  would 
benefit  rather  than  undermine  motivation  and  learning  merits  further  investigation.  Perhaps  by  the  time  students 
reach  middle  school  they  have  been  trained  to  value  grades,  especially  in  a setting  where  they  believe  they  are 
performing  well.  Even  so,  the  practice  may  still  focus  students’  attention  on  working  for  a grade  rather  than  learning 
so  as  to  satisfy  their  own  interests  and  desire  for  mastery. 

Factors  affecting  motivation.  The  factors  to  which  teachers  attribute  students’  motivation  are  likely  to  be 
the  ones  they  nurture,  while  factors  they  deem  irrelevant  may  be  unthinkingly  discarded.  For  example,  teachers  who 
believe  that  students  are  motivated  by  time  on  computers  may  skip  activities  such  as  whole  class  discussions,  even 
though  these  discussions  support  collaboration  and  problem-solving  skills.  Similarly,  teachers  who  do  not  believe 
students’  control  over  their  process  to  be  a factor  in  their  motivation  may  disregard  the  student-centered  nature  of 
the  program  entirely  and  direct  students’  work.  For  this  reason,  it  is  essential  that  researchers  identify  the  conditions 
under  which  students  are  most  motivated  to  take  ownership  of  their  work,  as  well  as  those  classroom  factors  that  can 
undermine  motivation,  and  communicate  their  findings  to  teachers  in  support  materials  for  these  programs. 

Floundering.  By  their  very  nature,  student-centered  learning  environments  create  opportunities  for 
students  to  pursue  unproductive  courses  of  action.  This  means  that  teachers  must  make  moment-to-moment 
decisions  about  how  to  respond:  whether  to  direct  students,  question  students  to  help  them  reconsider  their  efforts,  or 
to  permit  students  to  pursue  these  paths  until  they  recognize  a need  to  try  something  different.  Teachers  are  likely  to 
have  to  make  such  decisions  numerous  times  in  the  course  of  a single  class.  Most  of  the  participating  teachers 
expressed  a fairly  high  degree  of  tolerance  for  floundering  and  believed  that  it  could  benefit  learning.  However, 
considering  the  frequency  and  speed  with  which  teachers  must  make  decisions  related  to  this  issue,  further 
investigation  is  necessary  to  determine  if  teachers’  stated  beliefs  actually  match  their  actions.  Also,  a better 
understanding  of  effective  practice  around  this  issue  is  needed  in  order  to  inform  preservice  teacher  education  and 
professional  development  programs. 

Belief  Systems.  The  results  reported  here  can  provide  some  groundwork  for  future  investigations  of  the 
relationship  between  teachers’  beliefs,  their  actions  while  implementing  programs  designed  to  be  student-centered, 
and  student  outcomes.  For  example,  our  data  suggested  that  participants  differed  in  which  of  two  conceptualizations 
of  the  teacher’s  role  they  held.  If  patterns  like  this  can  be  established  and  compared  in  terms  of  student  outcomes, 
this  research  could  help  to  inform  preservice  and  inservice  professional  development  about  best  practices. 
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Abstract 

Over  1 000  preservice  teacher  candidates  have  participated  in  this  research  effort  that  contributes  to  the 
development  of  a model  for  the  infusion  of  technology  within  preservice  preparation  programs.  This  paper 
specifically  focuses  on  the  initial  analyses  and  findings  from  the  attitudinal,  anxiety,  and  computer  use  data  that 
have  been  collected  to  meet  one  project  goal  of  a PT^  implementation  grant.  Four  attitudinal  factors  related  to 
computer  use  were  explored  and  data  are  reported  for  computer  anxiety  and  computer  use. 

Introduction 

During  the  Fall  of  2000,  a mid-sized  university  began  a research  effort  ultimately  focused  on  the 
development  of  a technology  infusion  model  for  preservice  teacher  education,  one  of  five  project  initiatives  funded 
by  a PT^  implementation  grant.  The  theoretical  basis  of  the  hypothesized  infusion  model  is  that  for  preservice 
teacher  candidates  to  effectively  infuse  technology  within  their  future  classroom  settings,  they  must  develop  and 
practice  personal,  professional,  and  pedagogical  skills  related  to  technology  use  during  all  phases  and  experiences  of 
their  preparation  program.  Luetkehans  and  Robinson  call  for  teacher  preparation  programs  to  view  the 
“responsibility  for  teacher  preparation  as  shared  among  a triad:  preservice  teacher — inservice  teacher — and  college 
faculty”  (2002,  p.  15).  The  five  project  initiatives  of  the  PT^  grant  project,  together,  are  aimed  at  that  very  target. 

We  hypothesize  that  the  attitudes  and  values  systems  of  preservice  candidates  regarding  technology  and 
their  role  as  future  teachers  will  influence  their  likelihood  of  developing  technology-infused  future  classrooms.  Hill 
and  Schrum  support  this  hypothesis,  “Each  of  these  elements — beliefs,  theories,  and  curriculum — help  to  shape  the 
methods  or  strategies  we  use  for  teaching”  (2002,  p.  25).  In  addition,  we  hypothesize  that  preservice  candidates’ 
attitudes  are  influenced  by  their  exposure  to  technology  infused  instruction  (both  in  higher  education  and  K-12  field 
experience  settings)  during  their  teacher  preparation  programs.  This  hypothesis  is  supported  by  the  National  Council 
for  Accreditation  of  Teacher  Education  (NCATE)  in  the  expectation  that  accredited  schools  of  education  “provide 
adequate  access  to  computers  and  technologies,  and  expect  faculty  and  candidates  to  be  able  to  use  technology 
effectively  as  a teaching  tool”  (NCATE,  2002,  p.  3).  Unfortunately,  there  is  continuing  evidence  that  many  teachers 
resist  using  technology  and  do  not  have  the  knowledge  or  skills  necessary  to  effectively  integrate  these  tools  into 
their  teaching  (Cuban,  2001;  Molenda  & Sullivan,  2002). 

One  piece  of  this  research  effort  has  been  the  collection  of  attitudinal,  anxiety,  and  computer  use  data  from 
1084  preservice  teachers  over  six  consecutive  semesters  beginning  in  the  Eall  of  2000.  The  class  rank  of  participants 
ranged  from  sophomore  to  senior.  Analyses  reported  here  combine  elementary,  middle  grades,  and  secondary 
preparation  programs.  All  data  used  in  these  analyses  were  collected  at  the  very  beginning  of  the  semester.  The  data 
collection  is  continuing  and  will  culminate  in  the  Spring  of  2003.  Attitudinal  data  along  with  self-reports  of  anxiety 
and  average  hours  of  computer  use  per  week  are  collected  from  each  preservice  candidate  at  two  points  during  their 
preparation  (ET  24X  and  ET  34X)  to  provide  a baseline  for  these  variables  and  to  allow  comparison/growth 
analyses.  Initial  analyses  of  the  data  collected  thus  far  are  reported  here. 

The  ET  24X  curriculum  is  designed  to  ensure  mid  - to  high-level  skill  development  with  a variety  of 
computer-based  applications.  The  course  is  heavily  project-based  using  constructivist  principles  and  allowing  for 
practice  opportunities  that  match  the  students’  academic  and  career  interests.  Preservice  candidates  typically 
complete  ET  24X  during  their  sophomore  year  while  concurrently  participating  in  lengthy  field  experience 
requirements  that  are  mostly  observation  and  tutorial  in  nature.  ET  34X  is  designed  to  focus  on  integration  strategies 
and  the  application  of  technologies  to  support  and  enhance  instruction.  This  course  is  a combination  of  basic 
instructional  design  and  case  study  practice.  ET  34X  is  typically  completed  in  the  senior  year  just  before  student 
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teaching  and  is  coupled  with  extensive  field  experience  requirements  that  are  generally  characterized  by  lesson 
planning  and  the  teaching  of  mini-lessons.  Both  courses  are  lab-based,  instructor-led,  but  supplemented  with 
extensive  web-based  materials.  Both  courses  are  also  required  in  the  elementary,  middle  grades,  and  secondary 
teacher  preparation  programs. 

Instrumentation 

The  Attitudes  Toward  Computer  Usage  Scale  (ATCUS),  developed  by  Popovich,  Hyde,  Zakrajsek,  and 
Blumer  (1987),  was  the  instrument  used  to  collect  attitudinal  data.  This  instrument  was  designed  to  “...assess  how 
people  react  to  using  computers  and  computer-related  mechanisms”  (Popovich  et  al.,  1987,  p.  262).  The  authors  of 
the  ATCUS  instrument  conducted  two  studies  to  test  the  internal  consistency  test-retest  reliability  for  the  initial  40- 
item  scale  and  a second  20-item  scale.  In  the  first  study,  correlating  the  ATCUS  with  two  other  measures  of 
computer  attitudes  tested  the  convergent  construct  validity  of  the  instrument.  The  resulting  internal  consistency  for 
the  ATCUS  was  reported  as  .88.  The  test-retest  was  reported  as  .84.  Additionally,  significant  correlations  were 
reported  between  the  ATCUS  and  the  other  two  measures  of  computer  attitudes  used  to  test  convergent  construct 
validity  (r  = .62,  p<.001  and  r = .52,p<.001).  The  second  study  used  the  20-item  scale  that  excluded  redundant 
items  identified  in  the  first  study.  The  reliability  for  the  20-item  scale  was  reported  as  .84  with  the  test-retest 
correlation  reported  as  .91. 

The  revised  ATCUS  includes  four  sub-scales:  negative  reactions  to  computers,  positive  reactions  to 
computers,  approval  of  computer  use  in  education,  and  comfort  with  familiar  computer-related  mechanisms.  The 
third  and  fourth  sub-scales  were  reworded  slightly  from  the  Popovich  et  al.  (1987)  original  descriptions  to  clarify  the 
directionality  of  respondent’s  reactions  toward  computers.  The  description  of  sub-scale  three  was  changed  from 
“computers  and  children/education”  to  “approval  of  computer  use  in  education”;  the  description  for  sub-scale  four 
was  changed  from  “reactions  to  (familiar)  computer-related  mechanisms”  to  “comfort  with  familiar  computer- 
related  mechanisms.”  Items  were  not  modified.  Ranges  for  each  of  the  sub-scales  (factors)  are  provided  in  Table  1. 

Additionally,  two  self-report  items  were  added  to  the  survey  instrument  that  asked  preservice  students  to 
estimate  the  number  of  hours  per  week  that  they  use  a computer  for  any  purpose  and  to  rate  their  anxiety  level  (on  a 
scale  of  1 to  5,  with  1 = “very  anxious”  and  5 = “not  anxious  at  all”)  related  to  using  computers  and  computerized 
machines. 

Table  1.  Ranges  for  ATCUS  Factor  Scores 

Factor  1:  Negative  reactions  to  computers  [range:  7 (low)  to  49  (high)] 

Factor  2:  Positive  reactions  to  computers  [range:  5 (low)  to  35  (high)] 

Factor  3:  Approval  of  computer  use  in  education  [range:  5 (low)  to  35  (high)] 

Factor  4:  Comfort  with  familiar  computer- related  mechanisms  [range:  3 (low)  to  21  (high)] 


In  this  study,  the  scoring  of  ATCUS  was  inverted,  where  appropriate,  to  be  consistent  with  the  factor 
descriptions.  Popovich  et  al.  (1987)  commented  about  the  ATCUS  instrument,  "In  order  to  make  it  compatible  with 
the  previously  developed  Zoltan  and  Champanis  scales,  higher  scores  on  the  ATCUS  indicate  more  negative 
reactions  to  computers”  (pp.263-264).  The  descriptions  of  Factor  2,  Factor  3 (as  reworded),  and  Factor  4 (as 
reworded)  suggest  the  respondant's  degree  of  positive  reaction  toward  computers  — not  negative,  as  specified  by  the 
instrument  authors.  In  an  effort  to  clarify  the  meaning  of  high  and  low  scores  for  each  of  the  factors,  the  scoring  of 
the  following  questions  was  inverted:  1,  3,  6,  8,  9,  10,  11,  14,  17,  and  19.  For  those  questions,  the  Likert -scale 
responses  of  1 to  7 were  inverted  by  multiplying  the  value  by  -1,  then  adding  8. 

Analyses 

Mean  scores  for  each  of  the  factor  sub-scales  were  calculated  by  semester  for  ET  24X  and,  separately,  for 
ET  34X  (see  Table  2).  Means  for  each  factor  are  graphically  depicted  in  Eigures  1 to  4.  This  paper  reports  data  for  a 
total  N of  1084  preservice  teachers;  583  for  ET  24X  and  501  for  ET  34X.  A mitigating  variable  that  requires  further 
statistical  exploration  is  the  impact  of  the  extensive  curriculum  revisions  that  have  been  implemented  in  both 
courses.  Implementation  of  the  heavily  revised  ET  24X  curriculum  began  in  the  Eall  of  2001.  This  became  a major 
pilot  effort  and  significant  curriculum  revisions  were  made  again  before  delivery  in  the  Spring  of  2002. 
Implementation  of  the  revisions  to  the  ET  34X  curriculum  were  phased  in  with  about  half  of  the  re-designed  course 
implemented  in  the  Summer  of  2002  and  the  fully  re-designed  course  implemented  in  the  Eall  of  2002.  Eall  of  2002 
is  also  serving  as  a pilot  delivery  of  this  redesign  with  additional  revisions  planned  for  Spring  of  2003. 

Analysis  of  the  mean  data  show  few  differences  across  semesters  or  between  ET  24X  and  ET  34X.  Mean 
scores  for  Eactor  1 are  clearly  on  the  high  side  of  the  range  for  negative  reactions  to  computers  indicating  an  overall 
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negative  affect  related  to  computers.  Additional  analyses  considering  other  demographic  variables  (e.g.,  age,  major) 
and  potential  impact  of  curriculum  revisions  are  necessary  before  drawing  conclusions  about  the  significant 
decreases  observed  in  the  Spring  of  2002  for  ET  24X  and  for  both  ET  24X  and  ET  34X  during  the  Summer  of  2002 
that  indicate  a much  lower  agreement  with  a negative  affect. 

Conversely,  the  mean  scores  for  Eactor  2 are  on  the  high  side  of  the  range  for  positive  reactions  to 
computers  indicating  an  overall  positive  affect  about  computers.  Mean  scores  are  not  significantly  different  across 
semesters  for  either  course. 

Mean  scores  for  Factor  3 indicate  an  overall  positive  attitude  toward  the  use  of  computers  in  education. 
This  factor  is  particularly  stable  across  courses  and  across  all  six  semesters  of  data  collection. 

The  mean  scores  for  Factor  4 indicate  a moderate  comfort  with  familiar  technology.  Means  are  not 
significantly  different  across  semesters  or  between  courses.  This  is  interesting  given  the  types  of  “familiar 
technology”  addressed  in  the  ATCUS  items  (e.g.,  grocery  scanning,  automatic  teller  banking). 


Table  2.  Means  of  ATCUS  Factor  Scores  by  Semester 


Semester 

ET  Level 

Factor  1 

Factor  2 

Factor  3 

Factor  4 

N 

Fall  2000 

24X 

38 

30 

26 

12 

142 

Fall  2000 

34X 

38 

31 

28 

12 

96 

Spr  2001 

24X 

40 

31 

28 

13 

127 

Spr  2001 

34X 

39 

30 

28 

13 

146 

Sum  2001 

24X 

43 

30 

28 

12 

15 

Sum  2001 

34X 

39 

31 

27 

10 

14 

Fall  2001 

24X 

37 

30 

26 

12 

112 

Fall  2001 

34X 

39 

30 

27 

12 

129 

Spr  2002 

24X 

15 

31 

28 

13 

170 

Spr  2002 

34X 

39 

30 

28 

13 

88 

Sum  2002 

24X 

16 

31 

27 

12 

17 

Sum  2002 

34X 

17 

30 

29 

14 

28 

Figure  1 . Negative  Reaction  to 
Computers 
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Figure  2.  Positive  Reaction  to  Computers 
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Figure  3.  Approval  of  Computer  Use 
in  Education 
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Figure  4.  Comfort  With  Familiar 
Technology 


Fall  Spr  Sum  Fall  Spr  Sum 
2000  2001  2001  2001  2002  2002 


□ ET  24X 

□ ET  34X 


Semester 


Computer  Use  and  Anxiety 

Participants  in  this  study  were  asked  to  estimate  the  total  number  of  hours  per  week  that  they  used  a 
computer  for  any  purpose  during  a typical  week.  Mean  values  are  reported  in  Table  3.  N-values  differ  due  to  no 
response.  Again,  these  means  are  quite  stable  over  the  six  semesters  of  data  collection  and  across  courses. 

Candidates  were  als  o asked  to  give  a Likert -scale  rating  to  their  anxiety  with  using  computers.  Mean  scores 
are  reported  in  Table  4.  N-values  differ  due  to  no  response.  Mean  scores  are  only  slightly  above  the  middle  of  the 
scale,  tipping  toward  the  “not  anxious  at  all”  end.  The  stability  of  these  means  also  lends  support  to  the  resilience  of 
attitudes  and  anxieties  with  technology  regardless  of  skill  development  and  curriculum. 


Table  3.  Average  Computer  Use  in  Hours  Per  Week 


Fall  2000 

Spr  2001 

Sum  2001 

Eall  2001 

Spr  2002 

Sum  2002 

ET  24X 

8.7 

8.4 

7.3 

8.4 

10.5 

7.6 

N= 

139 

125 

15 

112 

164 

16 

ET  34X 

8.7 

8.9 

8.4 

9.1 

7.8 

8.5 

N= 

92 

144 

14 

125 

84 

25 

Table  4.  Average  Computer  Anxiety  Self-Rating 


Eall  2000 

Spr 2001 

Sum  2001 

Eall  2001 

Spr  2002 

Sum  2002 

ET  24X 

3.8 

3.7 

3.4 

3.6 

3.8 

4.3 

N= 

139 

115 

10 

93 

169 

17 

ET  34X 

3.6 

3.9 

4.0 

3.7 

3.8 

3.8 

N= 

92 

125 

10 

116 

87 

24 

361 


Conclusion 

The  tendency  for  the  traditional  pedagogy  to  be  maintained  is  legendary  in  our  observation  that  most  often 
we  teach  the  way  we  were  taught.  If  we  are  to  achieve  any  level  of  reform  in  pedagogy  to  institutionalize  the 
infusion  of  technology  within  classroom  practice,  these  results  indicate  that  the  affective  issues  of  attitude  and 
anxiety  related  to  technology  use  must  be  considered  in  the  complexity  of  teacher  preparation.  Technology 
integration  as  part  of  teacher  preparation  cannot  continue  to  be  about  exposing  candidates  to  best  practices  and  skills 
development,  but  must  re-focus  on  infusing  technology  integration  strategies  and  skills  into  the  knowledge  base  of 
pedagogy.  While  we  generally  accept  that  social  conditioning  occurs  through  modeling,  there  are  discrepancies 
between  the  curricula  of  most  preservice  instructional  technology  courses  and  what  is  the  typical  practice  of  teachers 
in  American  schools  (Betrus  & Molenda,  2002).  Initial  analyses  of  these  data  support  serious  consideration  of  the 
effect  of  attitudes  and  anxieties  on  technology  use  that  is  directly  related  to  the  adoption  of  such  tools  within 
pedagogy.  To  meet  the  standards  of  performance  and  accountability  that  are  set  for  the  preparation  of  future 
classroom  teachers  and  to  impact  the  true  infusion  of  technology  within  teaching,  we  must  incorporate  the  affective 
domain  within  the  design  and  delivery  of  our  curricula  and  field  experiences. 
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Introduction 

Rapid  advancements  in  computer  technology  have  had  a profound  impact  on  what  happens  in  the  2T‘ 
century  classroom.  There  are  increasing  demands  for  teachers  to  use  technology  in  their  classrooms  (Chambers  & 
Stacey,  1999).  Studies  have  concluded  that  classroom  students  get  more  enjoyment  of  subject  matter  and  motivation 
is  increased  when  technology  is  added  to  the  instructional  strategies  of  the  classroom  (Pugalee  & Robinson,  1998). 
Even  with  these  findings,  Moursund  and  Bielefeldt  (1999)  discovered  a number  of  national  reports  indicating  that 
instructional  technology  was  not  frequently  integrated  into  classroom  instruction.  With  2.2  million  teachers 
expected  to  be  hired  in  the  next  decade,  Moursund  and  Bielefeldt  also  point  out  that  this  is  a great  opportunity  to 
infuse  new  ideas,  philosophies,  and  practices  of  using  technology  into  the  2T‘  century  classroom. 

Effective  use  of  technology  can  have  positive  effects  on  teaching  practices  (Pugalee  & Robinson,  1998). 
The  literature  indicates  that  technology  can  improve  teaching  quality  and  have  positive  effects  on  student 
achievement  ('Teachers  Speak  Out  on  Technology  in  the  Classroom.  Instructor  Survey  Results,"  1991).  However, 
teachers  must  be  competent  computer  users  themselves  in  order  to  impact  students’  proficiency  with  the  technology 
(Barker,  1994).  Results  from  a 1999  study  suggest  that  a single  course  in  instructional  technology  cannot  provide 
adequate  preparation  for  preservice  teachers  to  integrate  technology  into  their  classrooms  (Willis,  Thompson,  & 
Sadera,  1999).  This  research  was  intended  to  discover  if  intentions  about  future  teaching  practice  are  influenced 
even  if  the  teachers  do  not  have  the  necessary  skills  to  fully  integrate  technology. 

This  qualitative  case  study  investigates  the  following  research  questions:  How  do  preservice  teachers 
understand  the  effect  of  an  instructional  technology  course  on  their  future  teaching  practice?  And  secondly,  how  do 
preservice  teachers  experience  the  effect  of  the  instructional  technology  course?  This  study  was  designed  to  explore 
the  effect  an  instructional  technology  (IT)  course  would  have  on  the  ideas,  philosophies,  and  identities  of  these 
future  teachers  by  examining  their  perceived  affect  of  the  course  on  their  future  teaching  practice.  A part  of  the 
investigation  is  designed  to  gain  some  understanding  of  how  preservice  teachers  experience  the  effect  of  the 
instructional  technology  course. 

Teachers  who  have  positive  experiences  with  computers  are  more  likely  to  implement  technology-based 
lessons  in  their  own  classroom.  Preservice  teachers  need  an  assortment  of  opportunities  to  use  technology  in  order  to 
enhance  their  own  learning  opportunities  and  instructional  technology  development  is  critical  to  their  preparation 
programs  (Brennan,  2000).  It  has  been  noted,  however,  that  future  teachers  often  come  into  a teacher  training 
program  ill-prepared  in  basic  technology  skills  and  don’t  feel  any  pressure  or  need  to  learn  it  (Willis  et  al,  1999), 
ultimately  it  is  up  to  the  faculty  to  model  effective  technology  infusion  and  instill  the  importance  of  the  use  of  K-12 
classroom  technology. 

There  is  an  ironic  and  costly  contradiction  in  the  attempt  to  integrate  technology  into  education. 

While  evidence  of  the  educational  benefits  of  technology  abounds  and  investment  in  hardware  and 

software  has  dramatically  increased,  relatively  few  teachers  use  technology  regularly  in  their 

teaching  and  the  impact  of  computers  on  existing  curricula  is  still  very  limited.  (Zhao  & Cziko, 

2001,  page  22) 

Review  of  Literature 

Instructors  who  incorporate  computer  technology  into  their  classroom  teaching  can  serve  as  role  models  for 
preservice  teachers  and  increase  the  likelihood  that  they  will  consider  technology  as  an  option  for  their  own 
classrooms,  “strategies  to  enhance  teacher  experience  with  computer  technologies  could  contribute  to  the  formation 
of  positive  attitudes  and  self-efficacy,  and  in  this  way  positively  influence  teacher  adoption,  use,  and  modeling  of 
computer  technologies’’  (Pepper,  1999). 

If  teachers  are  to  incorporate  the  use  of  technology  into  their  everyday  teaching  practice,  then  they  must 
expand  their  technical  skills,  increase  their  competence,  and  learn  how  to  incorporate  computers  into  everyday 
classroom  teaching  (Halpin  & Kossegi,  1996).  Courses  such  as  the  basic  introduction  to  instructional  technology 
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are  just  a stepping-stone  for  students  to  reach  the  necessary  competency  levels  and  have  a positive  impact  on 
students. 

In  order  to  guide  educational  leaders,  The  International  Society  for  Technology  in  Education  (ISTE)  has 
proposed  a list  of  National  Educational  Technology  Standards  (NETS)  for  all  preservice  and  in-service  teachers.  All 
classroom  teachers,  ISTE  states,  should  be  prepared  to  meet  the  standards  and  performance  indicators,  which 
include  the  ability  to  demonstrate  a sound  understanding  of  technology  operations  and  concepts;  plan  and  design 
effective  learning  environments  and  experiences  supported  by  technology;  implement  curriculum  plans  that  include 
methods  and  strategies  for  applying  technology  to  maximize  student  learning;  apply  technology  to  facilitate  a variety 
of  effective  assessment  and  evaluation  strategies;  use  technology  to  enhance  their  productivity  and  professional 
practice;  understand  the  social,  ethical,  legal,  and  human  issues  surrounding  the  use  of  technology  in  PreK-12 
schools  and  apply  that  understanding  in  practice.  (ISTE,  2000)  The  challenge  for  teacher  preparation  programs  and 
K-12  school  leadership  is  to  find  cost  effective  ways  to  provide  technology  professional  development  that  will 
improve  educational  technology  skills  with  the  expectations  of  positively  effecting  student  achievement. 

“A  typical  school  spends  just  0 to  5 percent  of  its  annual  technology  budget  on  in-service  training;  therefore 
odds  are  that  teachers  must  learn  a lot  on  their  own  or  informally  from  colleagues”  (Geisert  & Eutrell,  2000).  Thus 
suggesting  that  teachers  must  achieve  basic  computer  skills  in  the  teacher  preparation  program.  School  communities 
are  investing  in  computers  for  the  classroom  but  it  would  appear  that  they  are  only  recently  becoming  aware  of  the 
need  to  educate  the  teachers  on  effective  ways  to  infuse  technology  into  classroom  learning.  Computers  are  not  the 
end  but  the  means,  they  are  just  tools  to  help  to  improve  student  learning. 

Technology  in  the  Classroom 

Technologies  are  developed  by  humans  as  a way  of  adapting  to  the  world  and  are  invented  to  solve  a 
problem  The  key  to  helping  teachers  to  use  technology  in  the  classroom  is  to  help  them  first,  see  that  there  may  be 
a problem  (possibly  deficient  student  learning)  and  next,  that  the  computer  may  be  a tool  that  can  help  adapt 
teaching  methods  to  meet  the  needs  of  the  students. 

“The  word  technology  derives  its  meaning  from  the  root  work  technique.  Making  wise  choices  about  the 
use  of  technology,  then,  depends  on  recognizing  the  ways  in  which  it  facilitates  the  things  that  humans  can  do.  The 
tools  that  a society  uses  structure  the  techniques  that  are  available  to  humans  to  shape  the  world  in  which  we  live.” 
(Norton  & Wiburg,  1998) 

“A  defining  technology  is  a technology  that  results  in  fundamental  changes  in  how  people  see  themselves 
and  their  world.  A defining  technology  serves  as  a filter,  structuring  and  directing  people’s  interpretations  of  their 
experiences... Today’s  defining  technologies  are  the  electronic  technologies.  Thinking  ‘through’  the  electronic 
technologies,  we  are  presented  with  alternate  means  of  communicating,  informing,  and  knowing”  (Sheingold  & 
Hadley,  1990,  5-6). 

Eor  the  purposes  of  this  paper,  technology  will  refer  to  the  three  areas  of  technology  that  directly  affect 
education;  computers,  multimedia  and  the  information  infrastructure  (Internet,  superhighway)  (Schwartz  & 

Beichner,  1999). 

Using  technology  in  the  classroom  is  not  ‘new’  for  students  or  teachers.  Record  players,  tape  recorders, 
filmstrip  projectors,  televisions,  VCR’s  and  overhead  projectors  have  been  a staple  of  the  American  classroom  for 
decades.  Computers  have  been  in  schools  for  twenty  years  or  more  and  billions  of  dollars  have  been  spent.  It  has 
not  been  until  the  late  1990’ s that  the  average  classroom  teacher  was  expected  to  use  the  computer  in  his/her  own 
classroom.  Until  recently,  the  computers  were  housed  in  a special  computer  room  and  supported  by  an  educational 
technology  specialist.  Student  computer  use  was  limited  to  a special  lab  time  or  a computer  class  much  like  physical 
education  or  music.  The  message,  early  on,  was  that  teachers  didn’t  need  to  know  how  to  use  the  computer,  just  as 
they  don’t  need  to  know  how  to  play  the  piano  or  run  around  the  track,  these  were  all  things  that  students  would 
learn  from  ‘specialized  teachers’. 

East  forward  to  the  2T'^  Century  and  much  has  changed.  Budget  cuts  and  community  pressure  have  often 
removed  the  ‘specialized’  educator  and  placed  more  demands  on  the  classroom  teacher.  Computers  are  at  the 
forefront  of  the  revolution.  Computers  have  been  moved  from  the  individualized  lab  and  placed  into  classrooms. 
Teachers  who  once  could  avoid  the  electronic  machines  are  no  longer  afforded  that  luxury.  Educational 
technologists  now  play  an  advisory  role,  frequently  in  a number  of  schools  within  a school  district,  and  they  often  do 
not  take  responsibility  for  teaching  students  directly.  The  age  of  technological  innovations  has  been  brought  into  the 
safe-haven  of  the  average  classroom. 

Willis  et  al.  (1999)  suggest  that  most  teacher  education  faculty  have  positive  attitudes  toward  technology  in 
education,  moreover,  they  agree  that  an  infusion  program  can  be  an  effective  way  of  modeling  technology 
integration.  The  common  problem  in  the  preservice  teacher  preparation  program  is  that  the  faculty  members  don’t 
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have  a strong  background  in  actually  integrating  technology  into  the  courses  that  they  teach,  thus  perpetuating  the 
inadequate  teacher  preparation  (Willis  et  ah,  1999).  Faculty  and  classroom  teachers  may  be  frustrated  and 
unmotivated  to  use  the  classroom  computers,  possibly  because  they  have  not  had  time  to  adjust  to  the  innovation  and 
how  it  will  affect  the  their  use  of  time  and  impact  their  beliefs  and  philosophy  of  education. 

Sheingold  and  Hadley  (1990)  presented  a report  about  teachers  in  grades  4 through  12  accomplished  at 
integrating  computers  into  their  teaching.  The  researchers  discovered  that  many  of  these  teachers  devoted  their  own 
time  to  learning  how  to  use  computers  and  received  local  support  for  using  them.  The  researchers  concluded  that 
teachers  who  can  effectively  infuse  the  use  of  computers  in  the  elementary  classroom  are  more  likely  to  challenge 
their  students  with  more  complex  materials  and  encourage  more  independence  in  the  classroom  (Sheingold  & 

Hadley,  1990). 

Methods 

This  research  employs  the  case  study  method,  however,  there  are  many  types  of  case  study  techniques 
ranging  from  purely  qualitative,  non-generalizable  techniques  to  others  that  can  assume  some  degree  of  quantitative 
reliability  through  statistical  methods. 

Creswell  (1998)  defines  a case  study  is  “an  exploration  of  a “bounded  system”  or  a case  over  time  through 
detailed,  in-depth  data  collection  involving  multiple  sources  of  information  rich  in  context.  (Creswell,  1998,  p.  61). 
The  descriptive  case  study  presents  a detailed  account  of  the  case  study,  not  “..  .guided  by  established  or 
hypothesized  generalizations  nor  motivated  by  a desire  to  formulate  general  hypothesis”  (Merriam,  2001,  p.  38). 
Qualitative  research  is  not  designed  to  test  concepts,  hypotheses,  and  theories,  but  rather  build  and  describe  that 
which  is  being  researched  (Creswell,  1998). 

Five  preservice  teachers  from  a large  university  participated  in  the  descriptive  case  study.  They  were  all 
female  ranging  from  19  to  44  years  of  age.  Four  of  the  participants  are  Elementary  Education  majors;  the  fifth 
participant  is  focusing  on  Adult  Education.  All  of  the  students  were  enrolled  in  an  introduction  to  instructional 
technology  course  meeting  twice  per  week  for  one  academic  semester. 

The  students  are  all  preservice  teachers  with  different  class  standings,  ranging  from  a sophomore  to  a 
graduate  student.  They  all  have  a variety  of  backgrounds  working  with  children  and  in  schools.  The  students  with 
more  experience  were  better  able  express  their  ideas  about  student  learning  and  their  personal  philosophies.  Those 
with  less  experience  may  not  have  had  opportunity  to  consider  the  questions  in  the  past  and  they  had  a tendency  to 
give  more  generalized  answers. 

The  course  is  taught  by  two  different  instructors  as  a 2.00  or  3.00  credit  course.  Both  instructors  were 
helpful  in  providing  access  to  their  classrooms  and  their  students.  The  five  participants  volunteered  to  partake  in  the 
investigation  after  a brief  in-class  presentation  introducing  the  purpose  of  the  study.  The  course  goals  and  objectives 
were  the  same,  the  sections  were  each  taught  in  the  same  classroom,  and  similar  material  was  covered  in  all  sections. 

The  course  syllabus  indicates  that  the  course  is  intended  for  students  interested  in  educational  applications 
of  the  microcomputer  (i.e.,  lesson  plans,  presentation  tools,  course  supplements)  and  is  not  designed  to  teach  web 
development,  spreadsheet,  database,  the  Internet,  and  World  Wide  Web  in  the  absence  of  this  context.  Students 
must  have  completed  a prerequisite  course  in  educational  psychology.  The  introductory  course  introduces  the 
microcomputer  and  its  educational  applications;  instruction  will  be  based  on  the  premise  that  participants  are 
novices.  The  goal  of  the  course  is  to  teach  preservice  teachers  how  to  operate  microcomputers  as  well  as  develop 
skills  needed  for  the  appropriate  application  of  microcomputers  in  education.  Students  must  become  familiar  with 
the  range  of  educational  computing  issues,  which  includes  the  use  of  computers  as  teaching  devices,  presentation 
tools,  learning  environments,  as  well  as  developing  and  evaluating  Internet  resources  (course  syllabus,  2002). 

Over  the  course  of  several  months  classroom  observations  were  supplemented  by  a series  of  interviews 
with  the  five  participants  to  focus  on  questions  related  to  their  current  philosophy,  teacher  identity,  beliefs  about 
how  children  learn,  and  the  influence  the  IT  course  may  have  on  their  future  teaching  practice.  Student  work  was 
examined  to  determine  the  level  of  progress  the  participants  gained  during  the  semester. 

Results  and  Discussion 

The  interviews  were  transcribed  and  coded  in  search  of  common  themes  and  ideas.  Erom  a review  of  the 
interviews,  observations,  and  student  work  there  were  several  themes  that  began  to  emerge.  The  respondents  were 
very  helpful  and  had  many  interesting  things  to  say  about  student  learning  and  their  personal  philosophy  of 
education.  The  preservice  teachers  showed  a great  interest  in  student  learning  styles  and  making  children  feel  at 
ease  in  a learning  environment.  I tried  to  probe  into  their  ideals  to  determine  if  they  believe  technology  has  a place 
in  their  teaching  and  learning  philosophies. 

Theme  1:  Participants’  expectations  focused  on  skill  level  despite  the  course  goal  of  educational  integration. 
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In  the  first  interview  most  of  the  participants  had  a focus  on  basic  skills.  Although  skills  should  be 
considered  as  important  to  personal  success  in  the  course,  they  are  not  listed  as  a prerequisite.  This  instructional 
technology  course  is  not  as  much  about  computer  skills  as  it  is  about  integrating  technology  into  the  classroom.  It 
seems  students,  however,  approach  the  course  in  a different  way.  Many  of  the  opinions  of  the  respondents  seemed 
dependent  on  their  perceived  strength  of  computer  skills  and  discussion  often  centered  on  this  perceived  self- 
efficacy  despite  the  stated  course  goals. 

Mary,  a graduate  student  in  Adult  Education  stated,  “I  think  computer  skills  are  more  important  to  get  the 
most  out  of  the  class,  the  volume  of  what  we  are  learning  in  this  class  is  introductory,  it  does  not  depend  on  if  you 
know  Maslow’s  theory  or  you  know  chunking  or  any  of  those  kinds  of  things.” 

Janet,  a senior  elementary  education  major  was  excited  about  learning  the  skills,  “I  know  a lot  of  stuff  and  I 
think  this  class  is  helping  me  learn  it.  I take  a lot  of  notes  [on  how  to  make  the  databas  es  and  the  different  things] 
and  hopefully  that  will  help  me,  at  the  end  I will  have  this  resource  to  refer  back  to.”  She  indicated  that  little  of  what 
she  wrote  was  related  to  education;  it  was  mainly  about  the  technical  issues. 

As  they  look  ahead  in  the  semester,  students  are  more  concerned  with  the  technological  skills  they  are 
going  to  learn  than  the  ways  in  which  they  can  apply  their  learning.  Nancy  indicated  her  excitement  about  web  page 
development;  “I  think  we  design  something  we  put  on  the  Internet,  a web  page  or  something.  I think  that  I would 
like  to  do  that  because  I don’t  know  how  to  design  a web  page.” 

Mary  is  a highly  competent  computer  user.  Her  resume  includes  many  hours  of  computer  instruction  to 
returning  adults  and  university  faculty.  She  is  looking  for  some  skill  development  to  fill  gaps  in  her  own 
knowledge,  however  she  was  able  to  see  the  opportunities  to  apply  the  information  exists.  “Even  if  it  is  technically 
lower  than  what  I need  there  are  little  fundamentals  that  you  miss  out  on  if  you  don’t  take  those  introduction  things.” 
She  admits  the  biggest  help  to  her  is  being  on  the  other  side  of  the  computer,  “I  have  learned  to  give  students  smaller 
bytes  of  information  and  more  white  space  on  the  page.”  But  she  quickly  returns  to  a focus  on  computer  skills. 
“There  are  a lot  of  things  I enjoy  about  his  class  but  they  are  mostly  the  technical  things,  like  the  artistic  things 
because  I am  always  looking  for  those  ways  to  make  something  look  more  appealing  to  the  person  that’s  reading  it.” 
As  an  interviewer  I was  aware  of  the  course  goal  of  focusing  on  the  educational  integration,  I asked  Ellen  if 
she  thought  computers  changed  the  way  people  learn  and  in  turn  would  change  the  way  she  taught  children  in  the 
classroom.  Her  response  was  mainly  skill  related  with  less  emphasis  on  implementation,  “I  learned  things  that  can 
help  me  as  a teacher,  like  today  we  learned  about  the  database  and  which  definitely  you  can  use,  you  can  make  up  a 
database  for  all  your  students  that’s  actually  kind  of  interesting  and  helpful,  we  did  transparencies,  I am  sure  we  are 
going  to  learn  PowerPoint.”  Her  thoughts  seem  more  specific  about  what  applications  they  were  going  to  learn  and 
more  general  about  using  the  computer  technology  as  a classroom  tool. 

Ellen  is  a self-proclaimed  computer-illiterate.  She  lacks  many  of  the  basic  techniques  and  is  using  this 
course  as  a method  to  learn  the  computer  skills.  “I  can  type  a paper  but  that  is  about  it.”  I got  the  impression  from 
our  conversations  that  Ellen  was  much  more  concerned  about  controlling  what  students  did  on  the  computer  than 
actually  learning  to  use  the  computer  for  the  benefit  of  the  class. 

I am  hoping  to  take  what  I have  learned  in  this  class  to  help  me  understand  what  the  kids  are 
doing,  I am  sure  they  know  a lot  more  than  I will  ever  know.  So  when  I get  into  class  I want  to  be 
able  to  understand  that  hey,  she’s  talking  to  somebody  else  in  Social  Studies  right  now  and  I want 
to  be  able  to  correct  that  and  I also  want  them  to  be  able  to  use  that  computer  as  an  extra  tool  or  to 
find  things  that  may  enhance  what  I am  teaching. 

The  course  syllabus  recommends  that  students  attend  workshops  offered  by  the  computer  center  to 
supplement  their  skill  development.  None  of  the  5 study  participants  attended  those  workshops,  in  fact,  some  were 
not  even  aware  that  they  were  available.  When  Ellen  was  told  that  the  workshops  were  offered  she  replied,  “Really? 

I didn’t  even  know  about  that.  I guess  I could  go  but  it  would  depend  on  when  they  had  them,  my  schedule  is  pretty 
packed  right  now.” 

The  struggle  will  continue  between  developing  student  skills  and  modeling  technology  integration. 
Moreover  it  appears  that  preservice  teacher  ability  with  technology  can  adversely  effect  the  teacher’s  perception 
about  the  need  for  technology  integration.  Studies  continue  to  find  that  university  faculty  in  programs  that  prepare 
teachers  are  unsure  of  what  to  teach  their  future  teachers  or  how  to  use  technology  in  their  instruction  (Gillingham  & 
Topper,  1999);  thus  adversely  influencing  their  beliefs  about  the  way  to  students  learn  and  the  best  way  to  meet  their 
needs. 
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Theme  2:  Participants  recommended  that  all  preservice  teachers  take  this  course. 

There  continues  to  be  debate  among  professionals  as  to  whether  preservice  teacher  programs  should  offer 
stand  alone  courses  in  educational  technology  or  implement  a model  that  demonstrates  the  integration  throughout 
preservice  training  (Carbone,  2000).  To  that  end,  many  colleges  and  universities  are  removing  the  course  from  the 
offered  curriculum  and  replacing  it  with  the  infusion  model.  The  technology  infusion  approach  places  a variety  of 
technological  aspects  within  each  course  in  a teacher-preparation  program.  “The  positive  aspects  of  technology 
infusion  include  long-term  exposure  to  technology  and  technology  modeling  within  subject-matter  courses. 

The  negative  aspects  of  technology  infusion  include  inconsistent  implementation;  inability  to  calculate  faculty  load, 
provide  faculty  instruction,  or  faculty  support;  and  the  invisibility  of  technology  knowledge  on  student  transcripts’’ 
(Gillingham  & Topper,  1999).  The  teacher  preparation  program  in  which  these  students  are  currently  enrolled  does 
not  require  any  computer  course,  this  IT  course  is  offered  in  another  department  as  an  elective  course. 

In  general  all  the  participants  found  the  course  a satisfactory  experience  and  felt  that  other  colleagues 
would  also  benefit  from  the  course.  Janet  describes  it  this  way: 

Yes  this  has  been  a pleasant  experience  and  yesterday  in  my  other  class  I told  everyone  to  take  it 
so  obviously  I am  an  advocate  of  this  class,  it  is  not  required  for  me  I just  taking  it,  my 
concentration  is  in  math  and  science  and  I asked  and  they  passed  if  for  me  for  it  can  count  as 
math. . .1  think  it  should  be  required  now-a-days  - I think  there  should  be  more  publicity  for  this 
class. 

Similar  comments  from  Mary  and  Nancy,  “I  really  like  [the  instructor’s]  class  a lot’’  and  “this  is  a really 
good  class’’  reinforce  the  positive  attitudes.  Further  discovery  shows  that  the  experience  was  not  dissimilar  for 
novice  computer  users  or  the  more  experienced,  each  was  able  to  find  the  benefits  of  the  class.  The  syllabus 
indicates  that  the  course  will  be  directed  towards  the  novice  computer  user  but  it  appears  to  be  able  to  challenge 
even  the  more  experienced  computer  users  by  requiring  a number  of  independent  projects. 

If  technology  courses  are  not  required  in  the  preservice  teacher  course  of  study,  I was  interested  to  know 
how  students  perceived  the  integration  of  technology  through  their  methods  courses.  Nancy  responded: 

They  always  emphasize  integrating  technology  but  I think  maybe  in  each  of  my  methods  classes 
we  go  to  the  technology  lab  once  and  that’s  it,  and  we  see  websites  and  things,  actually  one  class  I 
am  taking  this  semester,  we  have  gone  to  the  internet  quite  a few  times  and  looked  a different 
educational  websites  and  looked  at  them  at  home. . . I think  they  emphasize  you  to  use  technology 
and  they  take  you  to  the  computer  lab  once.  One  of  my  classes  actually  uses  technology  but  the 
majority  of  classes  we  don’t  actually  hands  on  use  technology  in  the  class,  they  always  say,  well 
we  welcome  it  if  you  do  it  and  you  can  do  a project  with  it  but  its  never  like  this  is  how  you  can 
teach  a lesson  with  technology. 

Wanda  is  a sophomore  elementary  education  major.  Her  background  in  educational  theory  and  practice  is 
weak,  as  would  be  expected  of  a so  student  early  in  her  program.  She  is  seeking  a minor  in  instructional  technology. 
This  is  a class  I have  to  take  for  that  [her  minor]  and  it  is  the  only  class  I look  forward  to  every 
week,  I learn  so  much  in  that  class,  every  single  thing  that  he  does  in  that  class  seems  to  relate  to 
what  you  are  going  to  do  as  a teacher  someday  and  I just  think  that  the  stuff  he  has  taught  us  is 
really,  really  important.  More  teachers  should  take  this  class. 

Their  suggestion  that  more  teachers  take  this  course  raises  some  concerns.  Mainly  it  must  be  examined  as 
to  whether  it  is  the  computer  skills  that  are  more  important  to  the  student  or  the  ability  to  apply  them  in  the 
classroom.  It  has  been  indicated  that  teachers  who  know  how  to  use  computers  are  more  likely  to  integrate  them 
into  their  teaching  practice,  thus  it  appears  to  be  a circle.  Those  that  take  the  class  improve  their  skills,  thus 
improving  the  likelihood  that  they  will  integrate  technology  use  in  their  own  classroom;  the  modeling  done  by  the 
instructor  becomes  secondary.  Although  students  are  more  likely  to  implement  technology  use  when  it  has  been 
modeled  for  them,  the  lack  of  the  basic  skills  can  undermine  any  enthusiasm  to  actually  try  it  on  their  own. 

Theme  3:  Parental  involvement. 

The  message  that  involving  parents  with  their  children’s  daily  lives  in  the  classroom  can  have  a positive 
effect  on  student  achievement  is  being  sent  loud  and  clear  to  these  future  teachers.  Including  parents  with  email  and 
web  pages  were  just  a few  of  the  suggestions  they  made.  Typically  parents  are  included  in  bake  sales,  fund  raising 
efforts,  shuffling  papers,  and  word -processing  chores.  Jones  (2001)  indicates  that  these  common  parent  activities 
traditionally  associated  with  parent  involvement  are  not  likely  to  have  much  impact  on  student  achievement.  She 
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suggests  that  parents  must  be  further  engaged  in  specific  academic  activities,  such  as  tutoring  and  direct  student 
interaction,  in  order  to  have  a positive  effect  on  student  learning. 

Family  participation  in  well-designed  at-home  activities  appears  to  provide  the  greatest  impact  on  student 
achievement  (Jones,  2001).  The  participants  in  this  study  were  very  interested  in  involving  the  parents  in  their 
classroom  activities  yet  none  of  the  preservice  teachers  were  aware  of  the  importance  of  the  at-home  connection. 
Keeping  parents  involved  in  the  daytime  class  activities  was  their  main  focus.  Janet  was  asked  about  parental 
involvement  in  the  clas  sroom, 

I am  a strong  advocate  of  parental  involvement,  even  if  the  parents  can’t  come  in,  then  maybe 
grandparents,  or  aunts  and  uncles,  just  somewhere  where  a lot  of  people  can  help  and  volunteer  in 
the  classroom  and  I think  a lot  of  parents  are  looking  for  that.  If  parents  work  all  day  they  can 
[become  involved]  through  letters  or  possibly  where  their  more  in  charge  of  making  sure  that  the 
party  is  planned  and  they  don’t  necessarily  come  in  for  the  party  but  at  night  when  they  are  home 
they  can  call  the  different  parents  and  make  sure  that  Tommy’s  mom  has  the  cups,  that  way  they 
are  in  involved  that  way. 

Janet  was  clear  about  her  beliefs  about  including  families  in  the  classroom  activities  but  she  was  unclear 
about  the  benefits  of  using  technology  to  help  her.  There  is  little  doubt  that  parental  involvement  can  help  improve 
school-family  relationships,  which  can  positively  affect  student -school  relationships.  New  teachers  are  being  taught 
to  think  about  all  kinds  of  families,  socio-economic  levels,  and  cultures.  This  is  a very  positive  trend,  yet  they  are 
not  being  shown  ways  to  communicate  effectively  with  these  families  using  a variety  of  mediums. 

In  a University  of  Minnesota  study  conducted  by  professor  Joe  Nathan  (2001),  1,100  Minnesota  principals 
and  superintendents  were  asked  how  well  new  teachers  were  prepared  for  the  realities  of  their  work,  only  25%  said 
new  teachers  were  well  prepared  or  very  well  prepared  to  deal  with  parents;  in  contrast,  73%  reported  new  teachers 
were  well  prepared  or  very  well  prepared  in  content  areas  (Jones,  2001). 

Wanda  is  aware  of  implications  of  parents  helping  their  children,  yet  technology  does  not  seem  to  affect 
how  she  thinks  about  communicating  and  interacting  with  parents. 

I think  the  home  environment  definitely  influences  how  they  learn  in  school,  you  know  the  kids 
who  come  home  and  their  parents  don’t  ask  them  what  their  homework  is  and  don’t  help  them  at 
all  they  obviously  are  going  to  learn  a lot  less,  their  learning  will  be  less  reinforced  than  kids 
whose  parents  sit  with  them  at  night  and  help  them  with  their  homework  and  everything  like  that,  I 
think  knowing  kids’  background  is  really  important. 

Including  parents  in  student  learning  is  a talent  that  many  teachers  build  with  experience.  Parents  can  be 
intimidating,  demanding,  and  frustrating,  but  they  can  also  be  helpful,  excited,  willing  participants  in  the  classroom. 
It  is  important  for  preservice  teachers  to  consider  their  inclusion  in  their  teaching  practice  and  also  note  the  ways 
that  technology  can  assist  in  this  process. 

Theme  4:  Although  students  were  provided  with  a number  of  skills  and  ideas  it  is  still  questionable  as  to  whether 
these  teachers  will  use  computers  in  their  own  classrooms. 

There  is  a concern  among  educators  that  the  preservice  teacher  training  programs  are  not  sufficiently 
teaching  students  about  the  effective  use  of  technology  in  the  classroom.  Student  attitudes  toward  working  with 
computers  while  in  training  are  important  indicators  of  their  eventual  use  in  their  professional  classrooms  (Pepper, 
1999).  The  goal  of  this  IT  course  is  to  ease  those  concerns.  With  the  wide  range  of  student  computer  ability  and  the 
variety  of  disciplines  represented  in  the  class  population,  it  is  no  wonder  that  the  goals  are  difficult  to  achieve. 

There  are  about  18  students  enrolled  in  the  course,  study  participants  indicated  that  only  a few, 
approximately  7,  are  education  majors.  There  is  a small  population  of  computer  science  majors  and  business  majors 
as  well.  Several  interviewees  expressed  that  these  non-education  majors  have  better  computer  skills  than  the  future 
teachers.  Studies  indicate  that  previous  computer  exp  eriences  correlate  highly  with  student  attitudes  towards  using 
computers  (Pepper,  1999).  Those  interviewed  indicated  that  their  ejperiences  in  this  class  thus  far  had  been 
extremely  positive  and  their  skills  had  improved,  however,  the  focus  is  on  the  skills  not  eventual  classroom 
implementation. 

Mary  indicated,  “I  might  use  some  of  the  things  we  learned  in  class.”  But  there  appears  to  be  little 
improvement  for  Ellen,  “I  still  hate  computers,  well,  I don’t  hate  them  but  I get  so  frustrated  sometimes  that  I don’t 
know  how  to  do  things  or  that  I lose  it  and  I don’t  know  where  it  went  and  I don’t  know  how  to  get  it  back.” 

When  participants  were  directly  asked  what  they  would  do  to  incorporate  computers  into  classroom 
instruction  many  stumbled.  Janet  states:  “I  think  if  somebody  asked  me  like  what  would  you  do,  I might  say  like 
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type  their  work  and  make  it  iook  nice  and  put  it  outside  or  maybe  buiid  a book  or  something  iike  that  and  more 
formatting  stuff.” 

Eiien  aiso  echoes  with,  “You  can  type  papers,  I can  give  you  time  to  write  a journai  or  something,  iike  I 
made  a movie  on  a zip -disk,  I couid  have  them  iook-up,  weii,  I am  trying  to  think  of  something  eise,  there  has  to  be 
more  things.” 

Teachers  have  a whoie  scheduie  of  things  to  do  during  the  schooi  day.  The  frustration  of  not  truiy  knowing 
what  to  do  with  computers,  accompanied  by  an  attitude  that  they  may  not  be  necessary,  often  ieads  a teachers  to 
spend  their  time  eisewhere.  Eiien  has  an  oider  brother  who  is  currentiy  a ciassroom  teacher.  He  took  this  same  IT 
course  a few  years  ago  but  dispiays  a noticeabie  iack  of  computer  skiiis.  Eiien  did  not  seem  concerned,  “He  [my 
brother]  is  a good  teacher  and  I don’t  think  that  not  knowing  technoiogy  puts  him  at  a disadvantage,  he  wiii  just 
have  to  go  to  another  source  to  get  the  information.” 

Research  has  indicated  that  teachers  struggie  to  transfer  their  personai  computer  skiiis  into  their  ciassroom 
teaching  (Cesarone,  2000).  If  teachers  modei  the  teaching  behaviors  they  have  observed  then  it  is  up  to  the 
instructors  of  the  teachers  to  become  more  proficient  at  computer  integration. 

During  ciassroom  observations  I spent  time  watching  the  interaction  of  students  with  each  other  and  the 
instructor.  I was  not  concerned  with  the  content  of  the  iesson  as  much  as  I wanted  to  see  how  technoiogy  use  was 
integrated  into  the  ciassroom  instruction  and  what  kind  of  modeiing  was  demonstrated.  I found  that  the  majority  of 
the  time  was  focused  on  skiii  deveiopment  mainiy  working  on  an  assigned  project.  Thus  the  eventuai  conciusions 
that  students  drew  from  the  experience,  such  as  skiii  deveiopment  is  more  important  than  integration,  was  most 
iikeiy  a direct  response  to  that  which  was  witnessed  in  the  ciassroom  and  the  actions  of  the  instructor. 

Two  groups  of  three  and  four  members  respectiveiy,  compieted  the  work  I evaiuated;  oniy  one  member  of 
the  group  was  invoived  in  this  project.  It  was  difficuit  to  determine  what  kind  of  influence  the  participant  had  in  the 
preparation  of  the  project.  In  one  case  in  particuiar,  I know  that  the  group  contained  at  ieast  two  computer  science 
majors  with  greater  computer  experience  and  skiiis  that  may  expiain  the  compiexity  of  the  project.  The  oniy 
projects  that  I reviewed  done  by  one  person  was  produced  using  a word  processing  product,  which  cannot  give  me  a 
true  sense  of  what  she  can  do  with  technoiogy  integration. 

During  the  interviews  Eiien  expressed  concern  on  a number  of  occasions  that  her  work  was  not  as  weii 
done  as  her  computer  science  ciassmates.  She  was  showed  a serious  concern  over  the  quaiity  of  her  work  and 
contributed  her  iack  of  skiii  to  her  apprehension  to  use  the  technoiogy  in  a ciassroom.  “I  see  their  projects  and 
whoa,  I can’t  do  that,  I don’t  think  I can  ever  do  that.  When  students  see  that  kind  of  stuff,  iike  on  video  games  and 
stuff,  why  wouid  they  be  impressed  with  just  a bunch  of  coiored  words  on  the  screen?  No,  I have  a iot  to  iearn  yet.” 
If  teachers  don’t  feei  the  urgency  to  iearn  more  compiex  techniques  it  seems  they  often  ignore  the  technoiogy  option 
opting  to  spend  their  time  on  other  things  on  which  they  have  demonstrated  a higher  competence. 

Conclusion 

It  was  not  untii  the  course  was  in  the  finai  weeks  that  students  were  just  beginning  to  feei  comfortabie 
enough  with  their  computer  skiiis  to  pay  attention  to  the  purpose  of  the  course,  which  was  intended  for  students 
interested  in  educationai  appiications  of  the  microcomputer  and  not  to  basic  computer  skiiis.  Two  of  the  preservice 
teachers  are  participating  in  their  student  teaching  experience  in  the  faii  of  2002.  Euture  research  couid  inciude 
foiiow-up  interviews  with  them  when  the  experiences  are  compieted  to  determine  if  any  of  their  expectations 
regarding  technoiogy  were  met.  In  many  cases,  teachers  recognize  their  own  teaching  practice  faiis  short  of  their 
personai  ideai.  Teachers  struggie  with  what  they  wouid  iike  to  do  and  what  they  can  reaiisticaiiy  accompiish  given 
the  resources.  In  addition,  ciassroom  teachers  often  perceive  the  practice  in  their  schooi  as  even  more  ‘traditionai’ 
than  their  own,  providing  iittie  motivation  to  try  new  approaches  in  the  ciassroom  (Kuehn,  i993).  If  the  cooperating 
teacher  is  not  motivated  to  innovate  in  the  ciassroom,  the  preservice  teacher  may  modei  simiiar  behavior  iong  after 
her  student  teacher  experience. 

I wouid  recommend  that  students  suppiement  this  course  with  workshops  offered  by  the  university 
computer  center  in  order  buiid  computer  skiiis  and  seif-assurance.  This  new  found  confidence  may  heip  preservice 
teachers  focus  on  the  appiication  of  the  skiiis  and  not  the  acquisition  of  the  skiiis.  The  students  are  excited  to  iearn 
these  new  skiiis  but  a short  semester  is  not  enough  time  for  students  to  deveiop  them  beyond  the  basics.  Eiien 
indicated  her  concern  about  using  an  unfamiiiar  computer  in  her  ciassroom  “There  are  probabiy  programs  on  there, 
you  just  have  to  find  them  or  you  have  to  know  how  to  use  the  computer  to  find  that  and  then  it  can  be  heipfui  in  the 
ciassroom.  Otherwise  it  wouid  probabiy  sit  there  untii  someone  couid  heip  me  use  it.” 

It  was  unreaiistic  to  assume  that  this  IT  course  wouid  have  an  effect  on  future  teaching  practice.  The 
students  in  this  study  did  not  have  a weii-deveioped  phiiosophy  of  education  thus  making  it  improbabie  that  any  one 
course  couid  have  a sustained  impact  on  their  beiiefs.  The  students  indicated  that  there  isn’t  enough  time  to  affect 
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their  practice,  however  the  course  did  provide  a number  of  good  ideas  for  implementation  in  the  classroom.  Janet 
stated:  “it  [the  course]  has  shown  me  a lot  of  new  things  and  how  you  can  incorporate  different  things  beside  Word 
into  the  classroom  - I don’t  think  a lot  of  teachers  know  this  stuff.’’ 

Technology  is  a growing  part  of  our  culture.  Students  are  exposed  to  computers  in  the  grocery  store,  the 
fast  food  restaurants,  and  in  their  own  homes.  Computers  provide  another  tool  for  teachers  to  use  to  present  multiple 
representations  of  concepts  in  order  to  stimulate  greater  learning  in  their  students.  It  is  the  job  of  the  colleges  and 
universities  to  give  our  preservice  teachers  the  tools  they  need  to  meet  the  challenge  of  engaging  students  in  their 
own  classrooms.  In  this  instance  offering  a 2.00  or  3.00  credit  instructional  technology  course  as  the  way  of 
improving  the  eventual  use  of  these  tools  in  future  teaching  practice  was  not  the  most  efficient  method  for  these 
participants.  The  results  of  this  short  case  study  also  indicate  that  the  course  had  no  effect  on  the  students’  ideas 
about  future  teaching  practice,  nor  did  they  fully  understand  the  impact  that  the  course  had  on  their  beliefs. 
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Abstract 

Computer  technologies  are  overwhelming  the  job  market  and  education,  influencing  public  opinion  about 
their  wants  and  needs.  Many  people  see  how  common  place  computer  have  become  and  feel  that  their  use  is 
necessary  to  accomplish  work  and  improve  learning.  However,  the  reality  is  that  they  have  been  subconsciously 
influenced  by  the  media,  school,  and  the  society-at-large. 

The  goal  of  this  qualitative  study  was  to  understand  the  cultural  conceptions  about  computer  technologies 
of  12  economically  challenged  Hispanic  women  in  an  adult  literacy  program.  Findings  indicated  that  students  (a) 
perceived  computers  as  important  and  necessary;  (b)  wanted  to  gain  computer  skills  to  be  competitive  in  the  Job 
market  and  to  help  their  children;  and  (c)  recognized  negative  and  positive  aspects  of  computer  technologies. 

Introduction 

Many  Hispanic  adults  in  poverty  face  the  difficulty  of  being  considered  illiterate  while  raising  their 
children  with  limited  educational  resources  to  achieve  higher  levels  of  education.  Hispanics,  many  of  them  recent 
immigrants,  hope  that  through  education,  their  families  can  move  up  in  society  and  improve  their  quality  of  life. 
Programs  that  focus  on  low-income  families  target  this  population,  offering  them  educational  programs  that  support 
their  performance  in  a society  where  education  is  considered  essential. 

A better  understanding  of  students’  learning  experiences  and  perceptions  is  needed  to  expand  the 
knowledge  of  academically  and  economically  challenged  Hispanics.  It  is  especially  important  to  conduct 
investigations  about  personal  experiences  of  adults  in  programs  intended  to  break  the  cycle  of  poverty  and  illiteracy. 

Theoretical  Background 

The  theoretical  background  for  this  paper  is  based  on  computer  penetration  at  home,  motivation  to  learn  to 
use  computers,  and  the  influence  of  the  media. 

The  presence  of  computer  technologies  has  been  impacting  the  job  market  and  education  while  influencing 
people’s  opinions  in  unexpected  ways.  The  media  promote  advances  in  technologies,  assuming  that  they  bring 
progress  to  people’s  lives.  For  example,  television  and  the  press  provide  evidence  for  the  conveniences  of  accessing 
the  Internet  from  home.  Barton  (1994)  argued  that  published  advertisements  “try  to  convince  readers  to  make  these 
new  products  part  of  their  lives’’  (p.  58);  furthermore,  such  advertising  makes  some  people  subconsciously  accept 
the  “cultural  myth  of  technology-as-all-powerful-and-good”  (Duffelmeyer,  2000,  p.  291). 

With  the  spread  of  computer  technologies  in  the  last  decades,  people  receive  positive  media  messages 
about  the  need  to  adopt  new  technologies,  assuring  them  that  computers  will  facilitate  their  work  and  their  children’s 
learning.  When  parents  see  these  positive  messages,  they  become  convinced  that  computers  will  help  their  children 
“to  remain  competitive  in  school  and  in  their  future  careers”  (Takayoshi,  1996,  p.  199).  On  the  contrary,  the  media 
also  promote  negative  aspects  of  computer  technologies  such  as  the  possibility  of  the  Internet  making  children  more 
vulnerable  to  pornography  and  the  promise  of  romantic  dates  that  could  threaten  their  personal  well  being 
(Takayoshi).  As  a result,  people  also  become  aware  of  the  negative  aspects  of  computer  technologies.  However,  the 
positive  impact  of  the  messages  about  computers  usually  outweigh  those  risks  because  the  pressure  of  the  media 
promoting  the  overwhelming  success  of  computers  in  education  and  business  is  magnified  in  its  influence  on 
people’s  opinions. 

Technologically  advanced  cultures  present  different  messages  through  the  media,  and  society-at-large  in 
positive  and  occasionally  in  negative  ways.  Those  messages  are  subliminally  internalized  in  people’s  minds. 
McLaren  (2003,  p.  203)  argued  that  the  dominant  culture  “is  able  to  manufacture  dreams  and  desires”  where  without 
realizing  it,  people  develop  a common  way  of  thinking  through  the  use  of  “signs,  symbols,  and  representations.” 
With  this,  the  impact  of  dominant  culture  “power  and  privileges”  (p.  203)  is  hidden  while  influencing  people’s 
opinions  in  expected  ways.  These  dreams  and  desires  are  created  subconsciously  in  the  minds  of  people  who  are 
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exposed  to  positive  images  of  computer  technologies  that  have  been  promoted  by  different  entities  as  being 
necessary  to  their  wants  and  needs. 

As  a result  of  the  influence  of  the  organs  of  media  and  society-at-large,  people’s  conceptions  about 
computer  technologies  are  socially  constructed  and  impacted  by  the  dominant  culture  in  which  they  live.  These 
adulterated  conceptions  become  part  of  individuals’  thoughts  and  “appear  in  a culture  as  ‘natural’  and  ‘normal’ 
because  that  which  is  not  articulated  cannot  be  argued  against  nor  resisted’’  (Takayoshi,  1996,  p.  198).  People 
perceive  computers  as  important  artifacts,  necessary  for  work  and  educational  environments. 

It  was  our  interest  to  examine  selected  Hispanic  women’s  cultural  conceptions  about  computer  technologies 
and  to  determine  what  aspects  of  the  media,  school  or  society-at-large  influenced  those  conceptions. 

Methodology 

The  goal  of  this  qualitative  study  was  to  understand  the  cultural  conceptions  about  computer  technologies 
of  12  economically  challenged  His  panic  women  in  an  adult  literacy  program.  This  study  is  part  of  a broader 
investigation  that  examines  students’  sociocultural  context  and  cultural  conceptions  about  learning  and  computers. 
The  research  question  addressed  in  this  paper  was  “What  are  the  participants’  cultural  conceptions  about  computer 
technologies?’’ 

Context 

The  study  was  performed  in  an  adult  literacy  program  housed  in  an  adult  learning  center  in  a small  city  in 
the  Southwest  of  the  United  States.  The  goal  of  the  program  is  to  improve  three  areas  in  the  students’  lives:  early 
childhood  development,  parenting  skills  and  adult  education.  The  program  focuses  on  low- income  parents  with 
children  younger  than  eight  years  of  age  and  that  has  an  education  need.  The  program  provides  childcare;  students 
mentioned  this  service  as  an  important  reason  that  the  program  works  so  well  for  them.  They  explained  that  they 
could  not  study  without  this  assistance  because  most  of  the  participants  have  several  children  under  eight  years  of 
age. 

As  part  of  their  education,  the  students  were  enrolled  in  a 45  -minute  computer  literacy  class  that  met  once  a 
week.  The  primary  researcher  observed  the  students  in  their  computer  class  during  a period  of  1 1 weeks.  It  was  the 
first  time  that  this  class  had  been  offered  at  the  center.  At  the  beginning  of  the  semester,  the  class  was  taught  in  the 
reception  area.  Later,  four  desktop  computers  were  acquired  and  the  class  was  moved  to  the  GED  classroom. 

Participants 

By  February  2002,  the  program  had  a total  of  25  students,  24  of  them  from  Mexican  origin.  From  those  24, 
23  students  spoke  Spanish  as  their  first  language.  For  this  study,  12  students  were  selected  to  be  interviewed.  The 
age  range  of  the  participants  was  from  22  to  36,  with  an  average  age  of  29.  During  the  first  seven  weeks  of  the 
study,  the  primary  researcher  conducted  interviews  with  four  teachers  of  the  adult  literacy  program.  The  teachers 
were  asked  to  recommend  students  who  could  offer  significant  insights  to  the  study. 

Student  composition  varied  during  the  time  of  the  investigation  due  to  continuous  turnover.  This  turnover 
presented  a challenge  for  the  researchers  when  following  some  of  the  students’  stories  from  the  beginning  to  the  end 
of  the  study. 

Data  Source  and  Analysis 

The  primary  data  source  for  this  study  was  a series  of  semi -structured  interviews  with  the  12  selected 
students.  The  initial  interview  consisted  of  seven  open-ended  questions.  One  of  those  questions  “What  do  you  think 
influenced  your  opinion  about  computers?’’  provided  the  fundamentals  of  this  paper. 

After  a preliminary  analysis  of  the  interviews,  six  students  were  selected  for  a second  interview,  and  from 
those  six  students,  three  were  selected  for  a third  interview.  The  follow-up  interviews  were  more  in  depth, 
broadening  and  clarifying  topics  from  previous  interviews  and  observations  while  asking  students  about  issues 
related  to  the  computer  class.  This  interview  process  resulted  in  rich  descriptions  of  students’  conceptions  about 
computers.  In  addition,  as  a result  of  the  prolonged  time  spent  on  the  location,  thick  descriptions  of  students’ 
sociocultural  context  were  recorded.  Observations  from  formal  and  informal  student  activities  were  also  discussed 
during  the  follow-up  interviews.  This  research  method  helped  the  primary  researcher  understand  the  problem  under 
study  from  the  participants’  viewpoints. 

The  interviews  were  audio  taped  to  ensure  “that  everything  said  is  preserved  for  analysis’’  (Merriam,  1998, 
p.  87)  and  immediately  transcribed.  The  data  were  reviewed  and  categorized  according  to  the  emerging  themes 
following  the  constant  comparative  method  (Strauss  & Corbin,  1990).  Open  coding  took  place  from  the  raw  data 
during  the  initial  examination  of  the  interview  transcripts.  Extensive  readings  were  done  to  the  first  interview,  the 
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primary  researcher  wrote  comments  and  notes  in  the  sides  of  the  transcripts.  The  same  procedure  was  performed  for 
the  other  two  interviews. 

From  this  initial  reading  of  the  interviews,  the  researchers  divided  the  data  into  categories  of  information 
according  to  the  preliminary  themes  found  in  the  literature  review.  Examples  of  these  themes  included  computer 
penetration  at  home,  motivation  to  learn  computers,  and  the  influence  of  the  media  on  students’  cultural  conceptions. 
Emerging  themes  from  subsequent  readings  were  added  to  the  preliminary  themes.  The  text  was  pasted  on  cards  to 
allow  them  to  be  sorted  according  to  the  categories.  To  answer  the  research  question,  the  cards  were  reexamined 
searching  for  keywords  derived  from  the  categories  using  word  processing  features. 

After  this  process,  the  researchers  identified  a central  category,  students’  conceptions  about  computers  and 
the  Internet.  An  examination  of  the  cards  was  performed  to  find  the  factors  influencing  those  conceptions.  The 
factors  were  narrowed  until  obtaining  the  main  influences  in  students’  opinions. 

Results 

The  results  of  the  research  question  addressed  the  cultural  conceptions  about  computer  technologies  of 
twelve  Hispanic  women  in  a family  literacy  program.  These  results  were  based  on  a qualitative  analysis  of 
interviews  with  the  participants.  Three  major  findings  were  that  the  participants — (a)  perceived  computers  as 
important  and  necessary;  (b)  wanted  to  gain  computer  skills  to  be  competitive  in  the  job  market  and  to  help  their 
children;  and  (c)  recognized  negative  and  positive  aspects  of  computer  technologies. 

Perception  of  Computers  as  Important  and  Necessary 

The  first  finding  is  related  to  the  participants’  perception  that  computers  are  important  and  necessary.  The 
participants  perceived  the  computer  as  an  integral  part  of  all  learning  environments.  They  established  a link 
between  learning  and  computers,  considering  the  computer  indispensable  to  learning  or  accessing  information  that 
would  lead  them  to  learning.  They  cited  as  the  main  reason  for  needing  computers  skills  the  belief  that  these  skills 
were  essential  for  progress  in  their  present  U.S.  culture.  Even  though  many  of  the  participants  did  not  own  or  use  a 
computer,  they  assumed  that  computer  skills  would  help  them  and  their  children  to  acquire  learning  and  to  excel  in 
school,  and  for  that  reason  these  skills  were  perceived  as  very  important. 

Most  students  mentioned  that  they  would  like  to  own  a computer  and  access  the  Internet.  Some  students 
owned  or  had  used  a computer,  but  they  used  it  only  for  basic  applications  such  as  paint,  word  processing,  and 
games.  Students  perceived  a need  to  possess  computer  skills;  for  examp  le,  Irma  said,  “now  almost  everything  needs 
to  be  on  computers  or  by  the  Internet,  everything.  I think  that,  yes,  we  need  to  learn  computers.  It  is  a good  thing 
for  us.’’  Earlier,  Irma  had  bought  a $600  computer  for  her  daughters.  Irma  said  that  she  never  used  it  because  she 
was  afraid  to  break  it,  but  she  ensured  that  her  daughters  knew  how  to  use  it.  Participants  like  Irma  were  convinced 
that  computer  skills  are  important.  They  accessed  or  observed  computers  in  the  learning  center  where  they  were 
studying  and  at  their  children’s  schools. 

Of  all  of  the  students  interviewed.  Carmen  demonstrated  the  greatest  interest  in  teaching  herself. 

Curiously,  with  only  a sixth  grade  education.  Carmen  learned  to  access  the  Internet  by  herself  after  her  husband 
bought  an  old  computer.  One  of  the  activities  that  she  enjoyed  was  searching  for  recipes.  Carmen  would  first  enter 
“Univision,”  the  Spanish  television  network,  and  from  there  she  accessed  other  websites.  Carmen  explained  that  she 
also  “logged  into  the  Internet  to  learn  English. . . When  I need  information,  I access  a website  where  I can  post  my 
questions.”  She  posted  a question  about  finding  a program  where  she  could  learn  English,  and  she  received  a 
program  to  learn  English,  written  in  English  and  Spanish,  in  the  mail. 

Computers  are  one  of  the  resources  needed  in  the  participants’  current  learning  environment  here  in  the 
United  States.  Because  participants  noticed  the  presence  of  computers  in  places  that  they  usually  visited  (e.g., 
medical  and  social  services,  school,  and  utility  offices),  they  assumed  that  computers  are  necessary  in  any  work 
environment.  Participants  stated  this  as  an  important  reason  why  they  and  their  children  need  to  learn  computers 
skills.  Students  cited  the  computer  as  important  and  necessary  for  their  future  and  for  a better  life.  Some,  however, 
recognized  that  they  had  other  financial  priorities  and  could  not  afford  to  buy  a new  computer.  As  Elba  explained, 
“There  are  many  Hispanics  who  struggle  to  have  something  good  in  their  lives.  They  are  probably  interested  in 
computers,  but  sometimes  we  have. . . .,  at  least  I have,  other  priorities.  There  are  other  things  first.  I would  like  to 
have  a computer,  but  it  is  a luxury  for  me,  something  that  I really  do  not  need.” 

Participants’  opinions  about  computers  were  influenced  by  other  factors  in  their  daily  lives:  the  visual 
media.  Television  impacted  participants’  conceptions  about  computers.  When  participants  watched  TV 
commercials  and  news  reports  about  computer  technologies,  they  s tated  that  they  would  like  to  own  a computer  and 
that  they  were  motivated  to  learn  computer  skills.  As  suggested  by  McLaren  (2003),  people  perceive  a need  for  a 
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computer,  and  they  want  to  have  the  artifact  that  they  see  on  television.  The  participants’  “needs  and  wants’’  were 
driven  by  the  influence  of  television. 

Interest  in  Gaining  Computer  Skills 

The  second  finding  is  that  the  participants  wanted  to  gain  computer  skills  to  be  competitive  in  the  job 
market  and  to  help  their  children. 

Competing  in  the  job  market 

As  mentioned  earlier,  students  thought  that  computer  skills  are  essential  for  the  working  environment 
because  they  observe  computers  in  the  offices  that  they  visit.  Students  generalized  by  saying  that  computers  are 
everywhere  and  assumed  that  being  computer  literate  would  open  the  doors  for  the  job  market.  Cynthia  explained, 
“you  find  computers  in  every  place  and  find  a better  job  if  you  know  computers,  because  stores  and  all  types  of  jobs 
have  computers.’’  Cynthia  thought  that  Hispanics  would  be  more  competitive  in  the  job  market  if  they  knew  about 
computers.  Irma  explained  how  she  did  not  accept  a position  of  office  assistant  in  her  church  because  she  did  not 
know  computers.  She  emphasized  that  she  always  told  her  daughters  of  the  importance  of  learning  to  use 
computers,  and  that  is  the  reason  she  bought  one  for  them.  Irma  lived  with  her  three  daughters  and  husband  in  an 
old  two -bedroom  mobile  home;  their  family  struggled  because  only  her  husband  worked,  and  like  most  of  the 
husbands  of  the  interviewees,  in  a low-income  job. 

Helping  their  children 

Like  most  parents,  participants  were  worried  about  the  well  being  of  their  children  and  being  able  to 
prepare  them  for  the  future.  The  participants  perceived  computer  skills  as  necessary  to  the  progress  of  their 
children.  Parents  stated  that  computer  literacy  is  an  essential  part  of  progress  in  life,  school,  and  job  environment. 
They  believed  that  if  their  children  possessed  computer  skills,  they  would  have  better  chances  in  school  and  later  in 
the  job  market. 

In  addition,  participants  emphasized  their  preoccupation  with  being  able  to  answer  their  children’s 
questions  when  they  asked  about  computers.  As  Elba  explained,  “when  they  [children]  come  and  say  something 
about  computers  you  do  not  react  with  ‘what  are  you  talking  about?’  ...  .if  we  know  about  computers,  we  can  tell  our 
children  and  help  them  to  progress  in  life.’’  Other  students  also  stated  this  as  an  important  reason  for  being  computer 
literate.  For  example,  Elsa  said,  “When  they  learn  in  school  how  to  use  a computer,  I will  be  familiar  with  what 
they  are  doing.  In  that  way,  I understand  more  when  my  children  use  the  computer.’’ 

Students  not  only  wanted  to  learn  computers,  but  also  wanted  to  improve  their  English  skills  because  of 
their  children.  They  believed  that  with  computer  skills,  along  with  improved  English  skills,  they  would  be  able  to 
help  their  children  with  homework.  Most  of  the  students  stated  that  learning  English  was  their  primary  motivation 
for  enrolling  in  the  literacy  program.  When  they  discovered  that  they  could  also  take  computer  classes,  they  became 
very  enthusiastic  and  even  more  motivated,  an  “unexpected  surprise’’  according  to  Elba. 

Most  of  the  mothers  were  aware  that  their  children  were  learning  computers  skills  in  school.  They  assumed 
that  if  their  children  learned  computer  skills  they  would  have  better  job  opportunities  and  would  excel  in  school. 
Cynthia,  Gina,  Irma  and  Miriam  shared  that  if  their  children  learned  computer  skills  they  would  be  better  equipped 
for  the  future  workforce.  In  addition,  participants  spoke  of  their  motivation  to  learn  computers  to  help  their  children 
with  their  homework.  Carmen  mentioned  several  times  her  interest  in  helping  her  toddler  son  in  the  future.  She 
explained,  “I  could  help  my  child  with  his  homework,  because  there  will  be  many  assignments  on  the  computer.  I 
am  very  interested  in  being  able  to  help  my  son  if  he  asks,  ‘Mother  help  me.’’ 

Recognition  of  negative  and  positive  aspects  of  computer  technologies 

The  third  finding  is  that  the  participants  recognized  negative  and  positive  aspects  of  computer  technologies. 
Participants  paid  special  attention  to  the  developments  of  computer  technologies,  but  more  specifically  to  the  reports 
of  the  benefits  and  dangers  of  the  Internet.  These  reports  were  presented  on  television  documentaries,  which  detail 
cases  of  child  pornography  and  risks  of  online  relationships.  The  researchers  had  not  considered  this  finding  during 
the  development  of  the  investigation.  It  is  an  issue  brought  up  by  most  of  the  participants  during  the  interviews. 

They  had  the  basic  knowledge  of  the  Internet  and  how  it  can  be  used.  When  they  heard  the  television  news,  they 
feared  that  their  children  might  become  involved  in  such  “negative’’  experiences.  In  addition,  they  reported  a great 
distrust  of  relationships  generated  online.  Students  cited  television  news  that  alerts  people  about  the  risks  of  online 
relationships  that  can  lead  to  crime.  Lisa  was  aware  of  these  issues,  even  though  she  did  not  own  or  had  not  used 
computers  before.  She  was  aware  of  the  potential  for  the  Internet  to  be  a corrupting  influence  on  minors,  as  she 
explained,  “girls  meet  boyfriends  through  the  computer  and  it  can  really  be  a trap  for  them.’’  Nilsa  had  another 
concern  about  the  negative  impact  of  the  Internet  on  children:  “many  people  say  that  on  the  computer  you  can  learn 


375 


how  to  make  a bomb.”  Similarly,  Gina  was  afraid  of  the  negative  influence  of  the  Internet  and  information  that 
children  can  access.  She  said,  “the  only  thing  that  I do  not  like  about  computers  is  that  children  can  see  dirty  things 
on  the  Internet.”  The  negative  comments  were  very  similar  but  the  main  concern  of  students  was  the  access  to 
pornography  on  the  Internet.  Because  of  the  events  they  heard  on  the  news,  many  of  the  parents  were  over- 
protective  with  their  children;  two  of  them  reported  not  allowing  their  children  to  visit  their  friends’  houses  by 
themselves. 

On  the  positive  side,  participants  recognized  the  advantages  of  the  computer  and  the  Internet.  Carmen 
searched  for  recipes,  information  about  Mexico,  and  resources  to  learn  English.  Elba  visited  the  library  with  her 
daughters  to  access  the  Public  Broadcasting  Station  website.  She  liked  her  daughters’  experience  with  online  games 
and  coloring  books.  Irma  explained  to  their  daughters  the  magnificent  special  effects  on  television  and  movies  using 
computers  and  specialized  software.  She  encouraged  her  girls  to  learn  computers  and  pursue  a career  where  they 
could  have  a good  income  “just  with  a click  of  a mouse.”  She  explained  to  her  daughters  the  importance  of  studying 
to  avoid  falling  into  dead  end  jobs  where  they  would  need  to  work  in  the  sun  like  their  father. 

The  students  were  aware  of  the  uses  of  computer  technologies.  Eor  example,  Mima  was  amazed  at  how 
easily  information  can  be  sent  “from  one  place  to  another  instead  of  using  the  telephone.”  She  also  believed  that 
computers  would  help  her  to  improve  her  reading  and  writing  skills,  areas  in  which  she  was  working  very  hard.  She 
was  trying  to  pass  the  General  Educational  Development  Examination  (GED),  having  only  finished  eighth  grade. 
Other  uses  mentioned  by  students  such  as  Miriam  were  that  computers  can  be  use  to  store  information  and  that 
information  can  be  retrieved  when  needed.  Eor  example,  Miriam  hoped  to  open  her  own  business  in  the  future,  and 
she  planned  to  use  a computer  to  manage  her  business. 

Participants  were  aware  of  positive  and  negative  aspects  of  computer  technologies,  but  the  positive 
prevailed  over  the  negative.  Those  students  who  did  not  own  a computer  shared  their  interest  in  having  one,  but 
they  explained  that  buying  a computer  did  not  fit  in  their  immediate  financial  priorities.  As  Lisa  explained  to  her 
older  son,  “with  the  money  that  I need  to  buy  a new  computer,  I can  save  it  for  the  down  payment  on  a house.”  Lisa 
and  her  family  rented  a one-bedroom  house;  one  of  her  wishes  for  the  future  was  to  buy  a home  of  their  own,  but  she 
also  wanted  to  buy  a computer  for  her  son. 

In  summary,  the  results  of  this  qualitative  study  showed  that  Hispanic  female  students  enrolled  in  this 
computer  literacy  class:  (a)  perceived  computers  as  important  and  necessary;  (b)  wanted  to  gain  computer  skills  to 
be  competitive  in  the  job  market  and  to  help  their  children;  and  (c)  recognized  the  negative  and  positive  aspects  of 
computer  technologies. 

Discussion/  Conclusions 

Hispanic  Women’s  Cultural  Conceptions  about  Computer  Technologies 

The  participants  in  this  qualitative  study  (a)  perceived  the  computers  as  important  and  necessary;  (b) 
wanted  to  gain  computer  skills  to  be  competitive  in  the  job  market  and  to  help  their  children;  and  (c)  recognized  the 
positive  and  negative  aspects  of  computer  technologies.  The  results  support  the  findings  of  other  scholars  that 
suggest  that  people’s  conceptions  are  socially  constructed  and  impacted  by  the  dominant  culture  through  the 
different  entities  of  the  society.  What  entities  and  experiences  helped  to  shape  participants’  conceptions?  Erom  the 
data  gathered  for  this  study,  we  identified  these  factors  that  shaped  or  influenced  participants’  opinions:  (a)  cultural 
background;  (b)  life  experiences  and  struggles;  (c)  visits  and  observations  to  other  people’s  work;  (d)  the  adult 
literacy  program;  (e)  children’s  experiences  at  school;  (f)  teachers  and  friends;  and  (f)  print  and  visual  media. 

We  distinguished  three  categories  derived  from  the  data  analysis.  The  participants’  cultural  conceptions  of 
computer  technologies  were  influenced  by  the  sociocultural  context,  school,  and  media;  these  three  as  pects  of 
students’  lives  impacted  their  conceptions. 

The  Sociocultural  Context 

The  participants  in  this  study  shared  their  own  historical  background,  language,  and  experiences.  Most  of 
them  also  faced  the  challenges  of  immigration  to  a foreign  country  where  they  were  labeled  as  minorities  and  had  a 
limited  knowledge  of  English.  Therefore,  the  participants  shared  a unique  sociocultural  context  that  influenced  their 
conceptions  about  computers.  Even  though  most  of  the  participants  had  a low  level  of  education  and  came  from 
low- income  homes,  they  recognized  the  impact  of  computers  and  the  Internet  in  their  current  society.  Eor  example, 
participants  were  aware  of  the  information  that  they  could  access  on  the  Internet  to  obtain  cooking  recipes,  health 
information,  learn  English,  and  activities  for  their  children. 

When  some  minorities  in  poverty  face  this  overwhelming  promotion  of  technologies  in  the  society,  they  are 
likely  to  become  aware  of  being  computer  illiterate.  Many  minorities  decide  to  buy  a computer  or  enroll  in  classes 
hoping  to  learn  and  become  computer  literate.  Selfe  (1999)  points  out  that  citizens  of  color  “continue  to  have  less 
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access  to  high-tech  educational  opportunities”  (p.  423),  making  them  less  computer  literate  and  decreasing  their 
share  in  diverse  computer-related  jobs.  The  present  study  focused  on  12  Hispanic  females  enrolled  in  a computer 
class  in  an  adult  learning  center.  Of  the  12  students,  1 1 were  born  and  raised  in  Mexico;  the  twelfth  student  had  a 
Mexican  heritage,  as  her  family  immigrated  to  the  United  States  two  generations  earlier.  Therefore,  the  position  of 
the  students  in  their  current  context  put  them  at  a disadvantage  when  compared  with  other  ethnic  groups. 

The  School 

Students  enrolled  in  the  adult  literacy  program  with  the  main  goal  of  learning  English  or  obtaining  their 
GED.  Hispanic  students  who  recently  immigrated  to  this  country  felt  that  they  would  be  better  able  to  integrate  into 
this  culture  if  they  could  communicate  effectively  in  English.  By  enrolling  in  the  program,  participants 
demonstrated  a great  level  of  commitment  to  the  needs  of  their  children.  This  was  their  main  motivator  to  enter  into 
the  program.  They  also  felt  honored  and  recognized  when  were  accepted  into  the  literacy  program,  because  it  has  a 
high  rate  of  applications  and  a long  waiting  period. 

The  computer  class  opened  a new  window  for  students’  future  and  they  felt  those  skills  will  help  them 
when  their  children  ask  questions  about  computers.  As  Miriam  explained,  “these  classes  that  we  are  having  in  the 
center  can  be  the  only  opportunity  for  most  of  us  to  learn  how  to  use  computers.”  Participants  were  hopeful  about 
the  future  of  the  computer  class  in  which  they  were  enrolled  in  the  adult  learning  center. 

The  Media 

In  United  States,  Hispanics  are  the  fastest  growing  group  of  citizens  of  color;  many  of  them  come  to  this 
country  in  a search  of  a dream,  but  when  they  arrive  here  they  face  the  reality  of  being  considered  illiterate  and  fall 
into  menial  low-paying  jobs.  As  noted  earlier,  all  12  Hispanic  women  interviewed  lived  on  a limited  budget.  Eor 
many  families  in  poverty,  television  is  their  main  source  of  information  and  entertainment  (ref).  Thus,  these  factors 
could  influence  their  participants’  perceptions  and  cause  them  to  develop  adulterated  cultural  conceptions  about  the 
need  and  desire  for  computer  technologies. 

As  cited  by  most  of  the  Hispanic  women  participants,  the  visual  media  is  the  source  of  their  knowledge 
about  issues  related  with  computer  technologies.  The  participants  perceived  computer  technologies  from  a 
hegemonic  point  of  view.  That  is,  they  watched  television  reports  about  computers  and  mentally  translated  the 
messages  based  on  their  current  context  (Hall,  1974).  In  the  United  States  the  use  of  computers  is  highly  promoted, 
and  computer  skills  are  perceived  as  essential.  Not  surprisingly,  participants  assumed  that  computers  were 
necessary  and  they  wanted  to  buy  one.  The  participants’  cultural  conceptions  about  computers  and  the  Internet  were 
consistent  with  the  messages  received  from  the  media.  Erom  television  reports,  viewers  perceived  computer 
advances  as  omnipotent  elements  in  education  and  the  job  market,  and  they  assumed  that  those  advances  would 
improve  their  learning.  McLaren  (2003)  explains  this  phenomenon  as  the  dominant  culture  trying  “to  ‘fix’  the 
meaning  of  signs,  symbols,  and  representations  to  provide  a ‘common’  worldview,  disguising  relations  of  power  and 
privilege  through  the  organs  of  mass  media  and  state  apparatus  such  as  school,  government  institutions,  and  state 
bureaucracies”  (p.  203).  Television  mainly  promotes  the  use  of  computer  technologies  as  a symbol  of  progress  in 
society. 

Even  though  most  of  the  students  had  no  experience  using  computers,  they  showed  a positive  attitude 
toward  them  and  perceived  computers  as  important  and  necessary  for  the  academic  and  future  careers  of  their 
families. 

Final  Thoughts 

Students  demonstrated  a great  commitment  attending  the  computer  class.  They  conveyed  their  interest  in 
having  the  class  last  more  than  45  minutes  or  be  offered  more  days  a week.  They  also  shared  their  concerns  about 
the  future  of  the  class,  improvement  of  the  facilities,  and  having  a computer  teacher.  Einally,  students  felt  honored 
to  participate  in  the  study  and  recognized  the  importance  of  these  skills.  During  the  computer  class,  most  of  the 
students  saw  the  primary  researcher,  who  is  bilingual  in  English  and  Spanish,  as  another  teacher,  and  they  felt 
comfortable  asking  questions  of  her  in  their  native  language.  This  practice  represented  a dilemma  to  the  primary 
researcher  in  deciding  the  level  of  involvement  she  should  have  in  the  class  and  with  the  students.  Outside  the 
classroom,  the  participants  saw  the  researcher  as  a friend  and  shared  with  her  many  dreams,  advice,  and  their 
experiences  in  this  country.  This  phenomenon,  reported  in  the  literature  as  an  ethical  consideration  in  data 
collection,  is  typical  during  interviews  that  may  improve  the  condition  of  participants  (Merriam,  1998).  We 
appreciate  all  the  participants  who  kindly  participated  in  this  investigation,  and  we  recognize  their  motivation  to 
improve  as  human  beings,  their  roles  as  responsible  parents,  and  their  commitment  to  learning. 
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Abstract: 

Thirteen  K-12  teachers  participated  in  a technology  integration  professional  development  course  that 
included  course  components  such  as  peer  modeling,  peer  collaboration,  and  reflection  in  an  authentic  learning 
context.  The  study  purpose  was  to  explore  how  teachers’  beliefs,  practices,  and  self-efficacy  changed  in  this 
learning  environment.  Preliminary  results  indicate  evolving  teacher  beliefs  and  practices  as  related  to  these  course 
components.  Results  also  indicate  a significant  increase  in  teacher  self-efficacy,  based  on  pre-  and  post-course 
survey  scores.  Self-efficacy  remained  positive  for  key  research  participants  through  study  completion  in  Spring 
2002. 

Introduction  and  Background 

While  literature  shows  that  barriers  and  challenges  to  effective  technology  integration  exist  even  among 
exemplary  users  (Becker,  1994),  it  may  be  possible  to  address  some  of  these  barriers  through  professional 
development  strategies.  For  example,  research  suggests  that  peer  modeling  and  reflection  may  be  effective 
strategies  to  move  teachers  along  the  technology  integration  continuum  (Bandura,  1997;  Dwyer,  1996;  Gilmore, 
1995),  including  moving  from  a traditional  to  a constmctivist  (or  integrated)  approach  to  teaching  (Grabe  & Grabe, 
1998).  Peer  modeling  of  effective  teaching  and  technology  integration  strategies  may  lesult  in  increased  teacher 
confidence  and  competence  (Bandura,  1997;  Gilmore,  1995;  Pintrich  & Schunk,  1996). 

Further,  providing  models  of  exemplary  technology-using  teachers  in  a staff  development  setting  may 
facilitate  changes  in  teacher  beliefs  about  technology  integration  through  structured  exploration  of  those  beliefs 
(Ertmer,  Gopalakrishnan,  & Ross,  2000).  Seeing  other  teachers  similar  to  oneself  in  a successful  technology 
integration  capacity  may  cause  one  to  examine  and  possibly  revise  beliefs  about  meaningful  technology  use. 

A related  professional  development  strategy  is  teacher  collaboration,  whereby  teachers  share  ideas  and 
strategies  through  discussion  with  computer-using  peers  (Dwyer,  Ringstaff,  & Sandholtz,  1991;  Hadley  & 
Sheingold,  1993).  Research  suggests  that  peers  provide  emotional  and  technical  support  in  the  classroom  (Dwyer  et 
al.,  1991;  Hadley  & Sheingold,  1993)  and  are  found  in  exemplary -user  environments  (Becker,  1994). 

Embedding  learning  about  technology  use  within  authentic  contexts,  including  active  learning  approaches 
such  as  problem-based  learning  may  contribute  toward  using  technology  as  a meaningful  tool  (Duffy  & Jonassen, 
1992,  Torp  & Sage,  1998).  Learning  in  authentic  contexts  may  contribute  toward  changed  teacher  practice  and 
increased  confidence  with  regard  to  technology  use  [Dwyer,  1996).  If  such  strategies  are  developed  and 
implemented,  we  may  increase  the  likelihood  that  teachers  will  use  classroom  technology  to  enhance  the  critical 
thinking  and  problem  solving  abilities  of  school  children. 

Research  Purpose 

Thirteen  K-12  teachers  participated  in  a professional  development  course,  which  included  components  such 
as  peer  modeling  (including  a CD-ROM  model  of  exemplary  technology-using  teachers),  peer  collaboration,  and 
reflection.  The  research  purpose  was  to  explore  how  this  staff  development  model  facilitated  changes  in  (1) 
teachers’  beliefs  about  technology  integration;  (2)  teachers’  technology  integration  practices;  and  (3)  teachers’  self- 
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efficacy  beliefs  for  incorporating  technology.  This  study  explores  how  teacher  beliefs,  practices,  and  self-efficacy 
changed,  given  this  professional  development  experience. 

The  study  was  guided  by  the  following  research  questions: 

1.  How  do  teachers'  beliefs  about  technology  integration  (e.g.,  role  of  the  teacher,  assessment)  change 
using  reflection,  collaboration,  and  modeling  in  a staff  development  program? 

2.  How  do  teachers'  technology  integration  practices  (e.g.,  assessment  strategies,  curricular  emphases) 
change  using  this  staff  development  model? 

3.  How  do  teachers'  self-efficacy  beliefs  about  integrating  technology  change  using  this  staff 
development  model? 

This  paper  will  address  preliminary  results  obtained  from  initial  and  post-course  teacher  interviews,  course 
assignments,  and  a self-efficacy  survey  instrument. 

Methodology 

Participants 

Thirteen  participants  in  the  technology  integration  professional  development  course  comprised  our 
purposive  sample.  Twelve  teachers  agreed  to  participate  in  the  study  at  various  levels  of  involvement.  Five  teachers 
participated  in  the  study  in  a limited  capacity,  agreeing  to  share  course  assignments  and  complete  surveys  specific  to 
self-efficacy  for  technology  integration.  One  teacher  agreed  to  participate  in  interviews,  surveys  and  to  share  her 
course  assignments.  Six  teachers  agreed  to  participate  in  semi-structured  interviews,  observations  and  a self- 
efficacy  survey,  as  well  as  share  their  course  assignments. 

The  participants  came  from  four  private  schools  in  a Catholic  diocese  in  a Midwest  city  and  represented  a 
range  of  grade  levels  and  content  taught.  School  demographics  of  the  course  participants  are  provided  in  Table  1. 
Teacher  information,  participant  demographics  and  their  classes  as  of  Spring  2001  are  provided  in  Table  2.  Updated 
information  for  Fall  2001  is  provided  in  Table  3,  as  some  of  the  key  research  participants  changed  grades  that  term. 


Table  1.  School  Demographics  for  Spring  2001  (Names  are  pseudonyms) 


Name 

Student 

Population 

Ethnic 

Makeup 

Grade  Level 

Hilltop 

285 

Approx. 

4% 

Pre-K  to  3 

Fairview 

293 

Approx. 

9% 

Pre-K  to  6 

Middleton 

133 

Approx. 

4% 

4-6 

Elm  Creek 

375 

Approx. 

3% 

7-12 
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Table  2.  Demographics  for  Research  Participants  - Spring  2001  (Names  are  pseudonyms) 


Teacher 

Name 

Research 

Participation 

Level 

School 

Yrs 

Tchg 

Teacher 

Degrees 

Obtained 

Grade 

Level 

Content 

Taught 

Class 

Size 

Classroom  and  Lab 

Computer 

Resources 

Caroline 

Extensive 

Hilltop 

23 

M.S., 

Elementary 

Education 

3rd 

All  subjects 

Began 
with  22; 
ended 
with  19 

Mac  and  PC 
available; 

3 classroom 
computers; 
printers,  digital 
camera,  scanner 

Clara 

Extensive 

Hilltop 

9 

B.S., 

Elementary 

Education 

and 

Language 

Arts 

1st 

All  subjects 

18 

5 computers; 
scanner,  printer, 

2 IBM  computers, 
1 1-Mac,  2 Macs 

Greta 

Extensive 

Hilltop 

18 

M.S., 

Education 

2nd 

All  subjects 

19 

2 1-Macs,  printer 

Teacher 

Research 

School 

Yrs 

Teacher 

Grade 

Content 

Class 

Classroom  and  Lab 

Name 

Participation 

Level 

Tchg 

Degrees 

Obtained 

Level 

Taught 

Size 

Computer 

Resources 

Kathy 

Extensive 

Eairview 

6 

B.S., 

Elementary 

Education 

3rd 

All  subjects 

Began 
with  18; 
ended 
with  19 

1 1-Mac,  1 Apple, 
2-Emates,  printer 

Eleanor 

Extensive 

Eairview 

10 

B.A., 

Elementary 

Education 

4th 

All  subjects 

26 

2 Apples,  2 Macs, 
and  a printer. 

Library : 2 1-Macs, 
approx.  10  E-Mates. 
Several  people  in 
building  to  ask  for 
help. 

Jennifer 

Extensive 

Elm  Creek 

15 

B.S.,  Math. 

7“’  and 

8th 

Math  and 
Geometry 

21 

Computer,  scanner, 
Eax,  printer;  access 
to  computer  lab 

Julia 

Extensive 

Elm  Creek 

5 

B.S.,  Home 
Economics 

7“’  and 

8<h 

Earnily 

Living; 

Clothing 

Construction 

17-18 

(Est.) 

Access  to  computer 
lab 

Amelia 

Limited 

Elm  Creek 

25 

B.A.,  Social 
Studies  Ed., 
M.A.,  U.S. 
History 

9-11 

U.S. 

History, 

World 

History 

Access  to  computer 
lab. 

Anne 

Limited 

Elm  Creek 

Masters, 

English 

Education 

11-12 

English 

Literature 

Access  to  computer 
lab. 

Ruth 

Limited 

Middleton 

47 

B.A., 

Elementary 
Ed.,  M.A., 
Elem. 
Admin. 

4th 

All  subjects 

2 1-Macs  in  class 
room;  28  Macs  in 
computer  lab 

Martha 

Limited 

Middleton 

13 

Masters, 
Admin,  and 

Principal 
(18  yrs) 

— 

NA 

I-Macs;  access  to 
computer  lab 
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Sarah 

Limited 

Middleton 

Supervision 
E.S.,  5“’ 

All  subjects 

2 1-Macs  in 

Elementary 
Ed.,  E.S.  in 
Special  Ed. 

classroom;  access  to 
computer  lab. 

Table  3.  Demographics  for  Key  Research  Participants  - Fall  2001  through  Spring  2002  (Names  are  pseudonyms) 


Teacher 

Name 

Research 

Participation  Level 

School 

Grade  Level 

Content 

Taught 

Caroline 

Extensive 

Hilltop 

¥ 

All  subjects 

Clara 

Extensive 

Hilltop 

2nd 

All  subjects 

Greta 

Extensive 

Hilltop 

Kindergarten 

All  subjects 

Kathy 

Extensive 

Eairview 

3rd 

All  subjects 

Eleanor 

Extensive 

Eairview 

4th 

All  subjects 

Jennifer 

Extensive 

Elm  Creek 

7“’  and  8“’ 

Math  and  Geometry 

Julia 

Extensive 

Elm  Creek 

7'*’  and  8“^ 

Eamily  Living;  Clothing  Construction 

Research  Design 

The  study  was  primarily  qualitative,  using  a case  study  methodology.  Quantitative  and  qualitative  data 
were  gathered  specific  to  self-efficacy  about  classroom  technology  use.  The  qualitative  data  were  gathered  to 
explore  and  describe  teacher  beliefs  about  technology  ntegration,  changes  in  technology  goals  and  teaching 
practices,  and  teachers'  self-efficacy  beliefs  with  regard  to  technology  use.  Quantitative  data  were  also  gathered  to 
examine  teachers'  levels  of  self-efficacy  with  regard  to  classroom  technology  use.  Data  collection  continued  during 
the  Fall  2001  and  Spring  2002  semesters  to  allow  time  for  changes  in  beliefs,  perceptions,  and  practices  to  emerge. 

Procedure 

The  Spring  2001  semester-long  professional  development  course  was  conducted  once  a week  in  three-hour 
sessions.  Using  electronic  and  peer  models  of  technology  integration,  the  course  was  designed  to  facilitate 
discussion  about  technology  integration  issues  and  to  identify  different  strategies  that  might  be  used  in  the  teachers’ 
classrooms. 

The  course  components  included  (1)  presentation  of  information  on  problem-based  learning  (PEL),  (2)  a 
series  of  facilitated  discussions,  (3)  presentation  of  the  electronic  models  with  related  discussions,  (4)  course 
readings,  and  (5)  the  collaborative  development  of  a technology-based  PEL  unit.  The  teachers  also  were  asked  to 
submit  reflections  on  the  various  parts  of  the  technology-based  PEL  unit. 

Instruments 

Primary  data  sources  included  initial  and  post-course  teacher  interviews,  observations  of  teacher 
classrooms,  surveys,  course  discussions,  and  course  assignments.  Initial  and  post-course  interviews  were  conducted 
to  learn  about  changing  teacher  visions,  beliefs  (e.g.,  teachers'  views  about  classroom  organization  and  management, 
assessment)  and  practices  specific  to  technology  integration,  as  well  as  changes  in  self-efficacy.  The  first  interview 
was  conducted  in  February  2001,  approximately  one  month  after  course  outset;  the  second  interview  was  conducted 
in  June  2001,  the  week  following  the  end  of  the  course.  A third  interview  was  conducted  mid-term  Fall  2001  and 
two  interviews  were  conducted  in  Spring  2002  (early  in  the  term  and  the  week  following  term  end)  to  continue  to 
explore  evolving  teacher  beliefs  and  visions,  classroom  practices,  and  changes  in  self-efficacy. 

Course  observations  were  conducted  to  observe  class  activities  and  teacher  discussions.  In  addition, 
classroom  observations  of  key  participants  were  conducted  on  a weekly  basis  to  observe  teacher  classroom  practices 
and  technology  use  through  Fall  2001.  Eecause  five  out  of  the  seven  key  participants  opted  to  implement  their 
technology  unit  in  Spring  2002,  biweekly  observations  were  also  conducted  during  that  time. 

Teachers  completed  a pre-  and  post-course  online  survey  relative  to  teacher  confidence  (self-efficacy);  this 
survey  was  also  administered  mid-term  Fall  2001  and  late  Spring  term  2002.  This  survey  instrument,  specific  to 
teacher  confidence  about  their  technology  integration  practices  (self-efficacy),  was  an  adaptation  of  an  instrument 
developed  and  tested  in  Fall  2000  (Ertmer,  Conklin,  Lewandowski,  Osika,  Selo,  & Wignall,  in  press).  The 
constructed  instrument  had  three  categories  (planning  for  classroom  technology  use,  implementing  classroom 
technology,  and  assessing  classroom  technology  use  and  impact),  with  10  items  each,  for  a total  of  30  items.  The 
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instrument,  based  on  a five-point  scale,  required  responses  ranging  from  “Very  confident”  to  “Not  at  all  confident.” 
Item  examples  follow: 

Relative  to  planning  for  technology  use,  I am  confident  that  I can: 

1)  define  teacher/student  roles  in  a technology-integrated  classroom. 

2)  plan  classroom  activities  that  facilitate  technology  integration. 

3)  plan  for  the  use  of  computers  with  large-group  instruction. 

Course  assignments  included  teachers’  visions  of  themselves  as  technology-integrating  teachers  (at  course 
outset  and  course  end),  development  of  a technology-based  PEL  unit  for  their  classrooms  and  accompanying 
reflections.  During  the  class  sessions,  teachers  engaged  in  group  dis  cussions  specific  to  teacher  ideas  about  what 
they  were  seeing  and  how  that  tied  in  with  their  views  and  goals. 

Quantitative  Data  Analysis 

Data  were  analyzed  for  the  12  course  participants,  based  on  pre-  and  post-course  (June  2001)  self-efficacy 
survey  results.  A repeated  measures  analysis  of  variance  was  also  conducted  post-course  for  study  participants  at 
different  times  through  Spring  2002  for  said  instrument.  Reliability  was  also  measured,  using  Cronbach’s  alpha. 

Qualitative  Data  Analysis 

Interview  data  (pre-  and  post-course)  for  the  seven  key  participants  and  pre-  and  post-course  descriptions  of 
technology  visions  (all  12  course  participants)  were  analyzed  and  coded  inductively  specific  to  teacher  beliefs, 
practices,  and  self-efficacy,  using  cross-case  and  within-case  analyses.  Examples  of  codes  include  “Contributors  to 
Learning,”  “Technology  Practice,”  and  “Contributors  to  Confidence."  This  coding  was  done  using  Atlas. ti®,  a 
qualitative  analysis  software  package  produced  by  Scientific  Software  Development. 


Results  and  Discussion 

Quantitative  Data  Results 

At  course  outset  (January  2001),  teacher  self-efficacy  scores  averaged  from  1.1  to  3.0,  with  an  overall  mean 
of  2.2  (using  a five-point  scale,  ranging  from  “Very  confident”  to  “Not  at  all  confident”),  with  a standard  deviation 
of  0.67.  At  course  end  (June  2001),  individual  teacher  scores  on  this  instrument  averaged  from  1.2  to  4.9,  with  an 
overall  mean  score  of  3.4  and  standard  deviation  of  1.0.  Using  a repeated  measures  analysis  of  variance  on  pre-  and 
post-course  data  (n=12),  an  F-value  of  44.31  was  obtained,  significant  at  0.0001. 

Teacher  self  efficacy  scores  remained  constant  mid-Fail  term  2001  (n=l  1),  with  a score  range  averaging  1.2 
to  4.9,  an  overall  mean  score  of  3.4,  with  a standard  deviation  of  1.2.  A repeated  measures  analysis  of  variance 
between  June  2001  and  mid-Fall  term  2001  showed  an  F-value  of  0.07,  which  was  not  significant  at  0.0001.  Data 
collected  from  key  research  participants  at  the  end  of  Spring  term,  June  2002  (n=7)  showed  a mean  score  range  of 
3.0  to  5.0,  with  a mean  of  4.2  and  a standard  deviation  of  0.87.  Repeated  measures  analysis  of  variance,  comparing 
mid-Fall  term  2001  and  June  2002  showed  an  F-value  of  9.96,  which  was  not  significant  at  0.0001.  Instrument 
reliability,  determined  using  Cronbach’s  alpha,  was  0.99  for  the  self-efficacy  instrument,  averaging  across  all  data 
collection  points. 

These  results  indicate  that  teacher  confidence  steadily  increased  from  the  initial  course  through  June  2002 
for  key  research  participants.  While  the  changes  were  not  significant  at  0.0001,  the  self-efficacy  scores  remained 
positive,  with  mean  scores  increasing  in  a positive  direction,  perhaps  indicating  the  effect  of  successful 
implementation  of  teacher  technology  units  developed  in  the  Spring  2001  course. 

Qualitative  Data  Results 

Preliminary  data  analysis,  based  on  all  interviews  and  descriptions  of  teacher  technology  visions,  suggests 
the  following  themes: 

• Contributors  to  learning  (including  components  of  the  staff  development  model)  viewed  as  useful  varied 
with  individual  teachers,  although,  the  more  “active”  types  of  learning — ^peer  models  and  collaboration, 
hands-on  experiences  working  on  the  PEL  technology  unit,  and  class  discussions  were  mentioned  most 
often  as  contributors.  Teacher  implementation  of  the  technology  unit  was  also  perceived  as  a contributor  to 
learning. 

• Overall,  at  course  end,  teachers  reported  an  increase  in  confidence  with  regard  to  technology  use. 
Contributors  to  this  increased  confidence  included  knowledge  increase,  hands-on  experience,  peer  support, 
and  feelings  of  accomplishment.  Confidence  indicators  included  experimentation  (willingness  to 
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experiment  with  technology  in  the  classroom)  and  increased  student  technology  use.  Implementation  of  the 
technology  unit  also  contributed  to  teacher  confidence,  as  did  student  motivation  and  enthusiasm. 

• There  are  indications  of  some  teachers'  revising  their  beliefs  with  regard  to  technology.  Teachers  indicated 
a new  awareness  of  and  appreciation  for  the  role  of  the  student,  the  role  of  technology,  and  the  role  of  the 
teacher  in  facilitating  learning  through  technology  use.  Contributors  to  revised  beliefs  included  a new 
appreciation  for  what  the  student  could  accomplish. 

• Teacher  technology  practices  included  increased  student  technology  use,  including  using  their  students 
to  teach  them.  Teachers  still  expressed  concern  about  assessment,  classroom  management  and 
classroom  organization  with  regard  to  technology  use.  By  Spring  2002,  teachers  were  indicating  that 
their  technology  practices  were  becoming  aligned  with  their  vision  for  technology  use. 

Contributors  to  Learning:  Consideration  of  the  Staff  Development  Model 

Course  components  valued  as  learning  contributors  varied  with  individual  teachers.  The  strongest 
contributors  to  learning,  based  on  post-course  interviews  with  those  teachers  who  participated  extensively  in  the 
research,  appeared  to  be  peer  collaboration  and  peer  models,  hands-on  experiences,  and  class  discussions.  Course 
reflections  also  led  to  new  insights  about  teaching. 

Clara,  Greta  and  Caroline,  all  teachers  at  Hilltop  Elementary  in  grades  1-3,  respectively,  collaborated 
together  on  their  technology -based  PBL  unit.  Of  the  contributors  to  their  learning,  Clara  and  Greta  both  mentioned 
peer  collaboration  as  being  valuable.  According  to  Clara: 

I think  the  biggest  thing  was  the  peer  collaboration,  because  Caroline  was  so  wise,  in-in  her 
knowledge... where  Greta  and  I were  lacking... even  the  confidence... So  that,  and  feeling  that  we  had 
someone  that  did  know  part  of  what  was  going  on;  and  it  was  neat  working  with  Caroline  because  we  found 
out  she  didn't  know  everything.  And  some  of  the  things  we  learned  together.  [Post-Course  Interview,  June 
6,  2001] 

Greta  also  appreciated  the  support  that  Caroline  gave  to  her.  Further,  it  appears  that  Caroline  (who  acted  as 
an  informal  technology  coordinator  for  the  school)  acted  as  a peer  model  for  Greta:  "...  I think  Caroline  was  such  an 
awesome... leader  for  us.  She  gave  us  a lot  of — but,  at  the  same  time,  a lot  of  the  things  she  did — we  just 
watched... And  so.  I’m  hoping  that  I can  take  what  I saw  her  doing  and  try  and  do  myself."  [Post-Course  Interview, 
June  4,  6,  2001] 

At  Fairview,  Kathy  (a  third  grade  teacher)  and  Eleanor  (a  fourth-grade  teacher)  also  collaborated  together 
on  their  project.  Kathy  found  working  with  Eleanor  helpful,  as  well  as  the  class  discussions:  "Well,  definitely  the 
class  discussions,  especially  the  one  that  we  had  on  the  security  issues  and  privacy...!  liked  that  one  a lot.... And 
working  with  Eleanor  helped  a lot... She  --she'd  always  have  so  many  great  ideas— I love  working  with  her."  [Post- 
Course  Interview,  June  6,  2001] 

Eleanor  found  the  hands-on  experiences  of  the  project  most  useful:  "...All  of  that  contributed.  And,  but 
actually  doing  it.... The  hands-on  was  the  most— yeah,  for  me...”  [Post-Course  Interview,  June  7,  2001] 

Through  course  reflections,  Julia  (a  and  8*  grade  teacher)  expressed  a new  insight  the  importance  of 
using  active  learning  approaches  in  her  classroom:  "What  better  way  to  teach  my  students  skills  for  life  than  to 
engage  them  early  in  PBL!  Could  I really  teach  them  anything  more  important?  [Technology  Vision  Revisited, 
April  24,  2001] 

It  is  interesting  to  note  that  these  teachers  overall  selected  more  “active”  methods  of  learning — hands-on 
experiences,  working  with  others,  participating  in  class  discussions.  While  these  components  were  mentioned 
frequently,  it  should  also  be  noted  that  reflections  and  course  readings  were  also  mentioned  as  learning  contributors. 
It  is  also  possible  that  the  electronic  peer  models  (which  were  used  in  the  course  a couple  of  times)  might  have  been 
a stronger  contributor  with  more  use. 

The  technology  unit  developed  in  the  course  was  implemented  by  six  of  the  seven  key  participants.  Julia  at 
Elm  Creek  implemented  her  unit  in  Fall  2001;  Jennifer  opted  not  to  do  so.  The  remaining  key  participants  at 
Fairview  and  Hilltop  spent  Fall  2001  preparing  their  students  for  the  upcoming  unit  and  other  class  projects  through 
skill-building  activities,  such  as  teaching  their  students  PowerPoint  and  KidPix.  Students  also  developed 
keyboarding  skills..  These  teacher  technology  units  were  implemented,  to  varying  degrees,  during  Spring  2002.  The 
implementation  of  these  technology  units  appeared  to  act  as  a catalyst  for  learning  for  some  of  the  teachers  —they 
learned  by  "just  doing  it."  [Clara,  Post-Course  Interview,  June  12,  27,  2002] 

Many  teachers  learned  by  watching  their  students  in  action.  When  asked  about  learning  contributors, 
Eleanor  responded: 
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The  kids. ..and  then  just  doing  it.  You  have  the  idea,  and  you  learn  so  much  just  by  doing  it.  And  the  kids 
help... the  checklists,  I thought,  was  very  helpful.  They  knew  exactly  what  to  do,  and— and  they  even  filled 
out  the  eval— evaluation  of  each  other,  without  me  even  having  to  remind  them  again,  and  so  I learned  that 
that  worked  really  well.  [Post-Course  Interview,  June  1 1,  2002] 

Kathy  learned  more  about  planning  and  classroom  management  with  technology: 

I— I don’t  know  that  I learned  it,  but  I had  to  remember  that  you  need  to  be  flexible... And  we  learned  that 
with  our  project;  we  tried  the  four  in  a group  and  it  did  not  work.  There  is  not  enough  work  for  four  people 
in  a group.  So  I learned  that... maybe  it  would  be  better  to  do  a test  run  with  our  project  with— before  you 
actually  start  it  with  the  kids,  maybe  would  have  been  a good  idea."  [Post-Course  Interview,  June  II, 
2002] 

Increased  Confidence.  Confidence  Contributors  and  Indicators 

Post-course  interviews  and  post-course  teacher  visions  collected  in  June  2001  indicated  increases  in 
confidence  for  six  out  of  seven  key  participants  with  regard  to  classroom  technology  use.  Contributors  to  this 
increased  confidence  included  knowledge  increase,  hands-on  experience,  support,  and  feelings  of  accomplishment, 
as  well  as  the  opportunity  (and  willingness)  to  experiment  with  technology  in  the  classroom. 

For  Kathy,  the  third  grade  teacher  at  Fairview,  an  increase  in  knowledge  also  related  to  her  confidence  in 
classroom  technology  use,  as  well  as  willingness  to  ask  for  support. 

I still  believe  that  technology  is  a wonderful  and  exciting  way  to  teach  and  motivate  students.  I feel 
somewhat  less  overwhelmed  by  the  amount  of  information  and  software  available.  I have  more  confidence 
in  myself  and  my  abilities  to  utilize  technology.  In  addition,  I no  longer  feel  embarrassed  to  ask  for  help 
when  I need  it.  [Technology  Vision  Revisited,  April  24,  2001] 

Peer  support  and  accomplishment  continued  to  contribute  to  confidence,  as  well  as  hands-on  experience. 
Greta  found  that  working  with  Clara  and  Caroline  helped  her  confidence  and  her  learning,  as  did  the  hands-on 
experience.  When  asked  about  confidence  contributors,  Greta  answered:  "Doing  it  with  Caroline... just  getting  that 
whole  page  together... turned  out  marvelously  because  of  Caroline,  I think... cause  having  her  in  the  group,  that 
made  me  be  able  to  say,  'Yeah,  I can  do  that,'  or  'How  did  you  do  it,  Caroline?'... Trying  to  get  so  that  I would  learn 
from  her."  [Post-Course  Interview,  June  4,  6,  2001] 

Kathy  believed  that  changes  in  her  classroom  could  be  attributed  to  her  increased  confidence  and  hands-on 
experience,  which  led  to  her  increased  technology  use  and  experimentation:  "I  guess,  mainly  my  confidence  has 
gone  up,  so  that  created  a change.  And  that  started  with  the  discussions  and... actually  getting  in  and  using  the 
computers,  and  finding  new  things  to  do  with  the  computers  myself."  [Post-Course  Interview,  June  6,  2001] 

As  their  technology  journey  continued,  teacher  confidence  with  regard  to  technology  use  increased  overall, 
due  in  part  to  hands-on  experiences,  support,  increased  knowledge,  and  feelings  of  accomplishment.  Indicators  of 
such  confidence  were  reflected  in  their  encouragement  of  and  confidence  in  increased  student  computer  use  and 
willingness  b experiment  with  technology.  The  chance  to  implement  the  technology  unit  also  contributed  to 
increased  confidence,  as  Eleanor  explains: 

Oh,  it's— it's— it's  definitely  increased,  because  once  you  see  something  that  you  planned  and  you're 
implementing  it--and  it  could  be--it  may  be  a struggle— a struggle  planning  it  and  a struggle  implementing 
it,  but  when  you  finally  get  it  done,  that  does  —that  increases  your  confidence  big  time,  I think.  And,  well 
now,  you  can  see.  I've  got  so  much  more  ideas  on  technology,  and  with  the  Power  Point,  I thought. . .it's 
taking  it  step  by  step,  but  now,  who  knows?  [Post-Course  Interview,  June  II,  2002] 

Contributors  to  teacher  confidence  included  the  students'  overall  excitement  and  motivation  in  using  the 
technology,  as  well  as  relying  upon  the  existing  support  structure.  When  asked  about  her  confidence  in  implementing 
her  goals  and  plans,  Kathy  responded:  "Well,  I guess  I’ve  learned  that  you  don’t  have  to  know  everything  in  order  to 
do  it.  That  you  can  draw  on  other  people’s  knowledge  and  things  as  you  go  sometimes."  [Post-Course  Interview,  June 
II,  2002]  Further,  specific  to  confidence  contributors,  Kathy  said:  "Just  the  success  that  we’ve  had. . .and  the— the 
motivation  for  the  students,  and  their  enjoyment  of  it."  [Post-Course  Interview,  February  II,  2002] 
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Confidence  was  indicated  at  the  end  of  Spring  term  2002  through  increased  technology  use,  an  increase  in 
planned  technology  use,  new  ideas  and  strategies,  and  the  expressed  desire  by  all  five  teachers  to  use  the  technology 
unit  again  in  the  future. 

Both  Clara  and  Greta,  who  initially  felt  a lack  of  confidence,  saw  a difference  by  Spring  term  2002,  in 
terms  of  their  confidence  in  implementing  goals  and  plans. 

Well,  I know  when  I started  the  class  I had--when  I started  the  class,  Greta  [pseudonym]  and  I were 
just. . .when  we  walked  out  of  the  room  the  first  time,  and  to  see  the  growth  from  there  to  here. . .like  my 
confidence  in  doing  it,  and  the  ideas  that  come  from  it.  So. . .1  just  see  a lot  of  growth  in  it.  [Clara,  Post- 
Course  Interview,  June  12,  27,  2002] 

Oh,  see  that -I  wouldn’t  have  probably  been  willing  to  take  them  [the  students]  up  to  the  Lab  by  myself. 
And— and  have  them  all  on  the  computer  and  think,  “Oh  my  gosh,”  but  we— I did  that  and 
found  out  that  it  worked  and  so  we  went  more.  [Greta,  Post-Course  Interview,  June  21,  2002] 

Changing  Teacher  Beliefs 

For  some  teachers,  existing  beliefs  about  technology  changed  or  were  enhanced  with  regard  to  technology 
use.  Clara  learned  early  in  the  term  that  technology  could  be  used  as  a tool,  rather  than  her  earlier  conceptions  of 
how  she  should  use  it  in  the  classroom. 

I've  learned,  mainly,  that  technology  is  a tool;  and  I think  before  I thought,  in  reading  these  things  and 
really  thinking  about  it,  I was  thinking  technology  more  as  a subject?... And  that— I teach  across 
curriculums...!  was  looking  at  technology  as  another  subject  area,  rather  than  seeing  it  as  a tool  that 
enhances  subjects  areas  I already  have,  so  that's  where  my  whole  thinking  has  changed,  and  I can  see  now 
that  this  can  be  a really  great,  effective  way  to  teach  some  of  the— my  areas  to  enhance  them...  [Initial 
Interview,  February  5,  2001] 

Although  her  beliefs  about  the  value  of  technology  had  not  changed,  at  course  end  Jennifer,  a grade 
Math  teacher,  spoke  of  revising  or  revisiting  her  beliefs  about  the  role  of  the  teacher,  specifically  teacher-directed 
learning  in  the  classroom: 

As  a result  of  developing  my  problem-based  unit,  I have  learned,  or  been  reminded,  that  technology  makes 
learning  more  student -centered.  I think  I would  be  very  naive,  even  wrong,  to  believe  that  all  students  learn 
best  in  a teacher-centered  classroom.  I have  found  myself  asking  "Do  the  students  really  need  me  to  show 
them  how  to  solve  a linear  equation  (any  math  concept  for  that  matter)  or  could  they  discover  this  on  their 
own?"  [Technology  Vision  Revisited,  April  24,  2001] 

As  of  the  end  of  Spring  term  2002,  teacher  beliefs  included  a awareness  of  and  appreciation  for  the  role  of 
the  student,  the  teacher,  and  technology  in  facilitating  technology  use.  As  Kathy  stated,  "I'm  the  guide... I don’t  like 
to  be  the  one— especially  in  computers,  where  so  often  sometimes  the  kids  know  more  than  you  do.  I don’t  like  to  try 
to  get  up  there  and  pretend  like  "Oh,  Fm--rm  the  one  who  knows  everything. . . let  me  do  this  for  you. . .And  I guess 
the  role  of  the  child  is  just  to  be  a learner— and  a leader  sometimes. . . " [Post-Course  Interview  June  11,  2002] 

Kathy's  new  appreciation  of  what  her  students  could  do  contributed  to  her  changed  beliefs.  "The  kids 
finding  so  many  things  on  the  KidPix,  I think— have  helped  to  change  my  ideas  about  what  they’re  capable  of  and 
what’s— what’s  available  out  there,  and  what  they  can  do  with  it,  I guess. ..To  let  them  just  take  it  on  their  own  and  go 
with  it.  They  just  really  surprised  me  with  how  much  they  were  able  to  do."  [Post-Course  Interview,  June  1 1,  2002] 
Eleanor  indicated  a renewed  appreciation  for  what  the  students  could  accomplish:  "It  was  a problem- 
solving process  for  the  kids,  also,  but  I was  amazed  at  how  much  they  took  over,  which  is  --is  great,  because  that’s 
what  I like  about  teaching  is  that. . .you  don’t  plan  to  go  in  and  tell  'em,  'See,  this  is  -this  what  I know,  and  this  is 
what  I want  you  to  do.'"  [Post-Course  Interview,  June  11,  2002] 

Teacher  beliefs  are  not  easily  changed,  and  traditional  roles  in  the  classroom  can  be  hard  to  overcome 
(Ertmer  et  ah,  2000;  Eullan,  1993;  Schrum,  1999;  Van  Haneghan  & Stofflett,  1995).  As  Schrum  stated,  "Teachers 
need  compelling  reasons  to  change  their  practice"  (1999,  p.  85).  In  the  above  examples,  it  appears  that  the 
components  of  the  model  may  be  helping  some  teachers  consider  and  possibly  revise  their  beliefs  with  regard  to 
classroom  technology  integration. 
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Teacher  Technology  Practices 

Perhaps  due  in  part  to  their  overall  increased  confidence  with  using  technology  in  the  classroom,  many 
teachers  tried  out  new  ideas  with  regard  to  technology.  In  doing  so,  they  addressed  challenges  such  as  classroom 
organization,  management  and  assessment,  as  well  as  time  constraints. 

Clara  described  a successful  use  of  technology  for  her,  one  that  she  had  only  tried  out  that  term  with  her 
students:  "Each  one  wrote  a little  paragraph  about  themselves... and  then  they  had  to  find  some  Clip  Art  or 

something  that  looked... that  showed—reflected  them... and  put  it  on  their--their  paper... and  print  it  out.  And  we  did 
get  that  done!  So  that  was  successful...!  think  because  the  children  enjoyed  it  so  much.  They  were  able  to  follow 
the  directions.  They  ended  up  with  a good  product...  and  they  had  fun."  [Post-Course  Interview,  June  6,  2001] 

Clara  also  had  her  students  working  on  researching  a dinosaur  project  on  the  Internet,  using  the  students  to 
help  each  other:  "What  I tried  to  do  there  is  put  someone  that  was  more  computer-literate,  someone  that  could  read 
well... with  someone  that  was  a little  bit  lower... so  they  could  work.  So  sometimes,  I didn't  have  as  many  high 
students,  so  sometimes  the  high  student  would  work  with  three  or  four... students  at  different  times.”  [Post-Course 
Interview,  June  6,  2001] 

Eleanor  had  her  students  teach  her  how  to  use  KidPix,  preparatory  to  using  her  technology  unit,  which  she 
had  developed  with  Kathy,  the  following  term:  In  doing  so,  she  and  her  students  used  KidPix  to  run  ads  on  the  VCR 
monitor  during  their  Mini-Economy  activity: 

Planning  our  unit,  we  were  planning  on  using  KidPix  and  I had  never  done  that  before... so  I did  go  ahead 
and  look  at  it  and  then  use  it  at  the  end  of  this  year,  so  that  I'll  be  more  able  to  jump  into  the  unit  next 
year.... That  was  a lot  of  fun.  And  here,  a couple  of  my  student  experts  helped  me  cause... they  had  it  at 
home  and  helped  me  how  to  use  it  and,  so  I had  no  idea  what  I was  going  to  do  with  the  kids,  but  we  ended 
up  doing  our  ads  with  our  Mini-Economy  on  there... with  the— the  music  and  the— the  slide  show  and  that— 
that  was  really  cool.  [Post-Course  Interview,  June  7,  2001] 

While  concerns  about  assessment  and  classroom  management  remained  as  issues  at  the  end  of  Spring  2002, 
many  teachers  found  that  their  ideas  about  technology  use  and  their  practice  were  coming  closer  together— teachers 
were  moving  closer  to  their  technology  vision,  aligning  more  with  their  beliefs  about  teaching  and  technology. 

Eleanor  spoke  early  Eall  term  2001  about  her  ideas  for  technology  and  how  that  tied  in  with  practice: 

We  got  the  whole. . .National  Weather  Service  up  there,  and  we  were  actually  watching  a front  come  in  and, 
sure  enough,  we  watched  it  right  out  the  window.  We  saw  it  come  in,  so,  I mean,  that — that’s  daily, 
everyday  stuff,  that  you’re  bringing  right  into  the  classroom. . .it’s  more  of  a. . .it's  something  that  is  there 
now. . .which  it — it’s  at  reach,  and  before  it  wasn’t.  [Post-Course  Interview,  September  20,  2001] 

When  asked  in  Spring  2002  how  her  vision  for  technology  comp  ared  to  practice,  Eleanor  responded,  " I 
think  it's  right  with  it... Actually,  it's...It— it--it's  intermingled... because  I think  maybe  my  vision  isn’t  very  lofty,  but 
it's—  it's  attainable  and  in  sight."  [Post-Course  Interview,  June  11,  2002] 

Clara  responded,  "Well,  I think  they're  probably  lining  up  [vision  to  practice] ...  and  running  parallel  to  each 
other.  Where— before,  my  vision  was  probably  right  on  track,  but  what  I was  doing  was  kind  of  faltering...  moving 
ahead  slowly  where  my  vision  was.”  [Post-Course  Interview,  June  12,  27,  2002] 


Conclusions  and  Future  Implications 

Impact  of  the  Staff  Development  Model 

It  appears  that  the  course  components  were  (perhaps,  not  surprisingly)  of  value  to  different  teachers  in 
different  ways.  The  hands-on  experience,  as  part  of  the  PEL  technology  unit,  were  mentioned  often  as  being  useful 
in  terms  of  increased  knowledge  and  confidence.  Peer  collaboration  was  particularly  useful  to  those  who  looked  to 
“leaders”  to  help  them  in  skill  building  and  developing  web  pages.  Reflections  were  useful  for  “rethinking”  where 
they  were  and  where  they  are  going.  In  some  cases,  reflections  may  have  contributed,  in  part,  to  changing  teacher 
beliefs.  Implementing  the  technology  unit  contributed  positively  to  teacher  confidence,  beliefs  and  practice. 

Overall,  the  impact  of  the  staff  development  model  appeared  helpful  in  terms  of  changing  teacher 
confidence,  beliefs  and  practices.  Participants  did  not  directly  say,  “This  component  helped  me  change  my 
practice.”  However,  in  the  sense  that  the  various  components  contributed  to  increased  knowledge  and  confidence,  it 
may  be  termed  useful.  Pre -course  profiles  indicated  that  teachers  varied  from  traditional  teaching  approaches 
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toward  more  integrated  approaches  to  classroom  technology  use  (Grabe  & Grabe,  1996).  While  further  analysis  is 
needed  to  confirm  these  initial  results,  this  may  lead  to  an  interesting  contrast  relative  to  changes  that  may  occur  as  a 
result  of  the  professional  development  course,  given  the  diversity  of  approaches. 

Further  Data  Analysis  Needs 

Analysis  of  interview  data  with  seven  teachers  appears  to  indicate  that  their  school  cultures  encourage 
collaboration  and  reflection,  with  some  teachers  also  referring  to  other  teacher  "models."  Given  the  existing  culture, 
it  will  also  be  interesting  to  see  how  teacher  beliefs  and  technology  integration  practices  are  facilitated  by  such 
cultures,  in  conjunction  with  (and  supported  by)  the  professional  development  model,  which  also  encourages 
collaboration,  reflection,  and  modeling.  Further  analysis  of  additional  data  sources  is  needed  to  confirm  these 
impressions.  Further  analysis  remains  to  be  done  on  course  data  and  observations.  While  preliminary  data  analysis 
has  focused  on  teacher  beliefs,  practices,  and  self-efficacy,  as  well  as  learning  contributors,  results  are  yet  to  be 
determined  in  greater  depth. 

With  regard  to  efficacy  data  analyzed  to  this  point,  results  are  somewhat  consistent  with  those  of  Stein  and 
Wang  (1988).  While  many  studies  have  examined  teacher  efficacy  in  professional  development  settings,  often  self- 
efficacy  is  measured  at  one  point  in  time  (Ross,  1995).  However,  research  has  shown  that  teacher  efficacy  may  be 
curvilinear  across  time  (Stein  & Wang,  1988).  According  to  Stein  and  Wang  (1988),  teacher  self-efficacy  may 
increase  during  a professional  development  course,  then  decrease  as  the  teacher  attempts  to  put  what  was  learned 
into  practice  and  then  later  increase  again  as  the  teacher  masters  what  was  learned.  It  may  then  be  useful  to  measure 
teacher  self-efficacy,  not  only  during  a professional  development  course,  but  also  as  a follow-up  (as  in  the  following 
term)  in  the  teachers'  classrooms.  In  this  case,  the  teacher  self-efficacy,  specific  to  the  key  participants  remained 
steady.  This  may  be  due,  in  part,  to  consistent  researcher  follow-up  in  Fall  2001,  fading  gradually  through  Spring 
2002,  until  data  collection  ended  after  Spring  term.  It  is  possible  that  such  follow-up  aids  teacher  self-efficacy. 
Further  research  is  needed  to  confirm  this  conclusion. 
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Abstract 

The  researcher  investigated  a telecommunications  project  among  Taiwanese  English  as  a Foreign  Language 
(EEL)  learners  and  Japanese  EEL  learners  in  which  they  practiced  English  language  and  discussed  cultural 
information  through  web -board  and  desktop  videoconference.  The  purpose  of  this  study  was  to  understand  the 
Taiwanese  EEL  learners'  perceptions  of  language  learning  through  distance  technologies  and  to  explore  the  turn- 
taking  and  intercultural  communication  issues  during  the  connections.  Areas  for  improvement  in  future  connections 
are  noted. 

Introduction 

This  study  investigated  a telecommunications  project  among  Taiwanese  English  as  a Foreign  Language 
(EEL)  learners  and  Japanese  EEL  learners  at  the  university  level.  The  goal  was  for  both  the  Taiwanese  students  and 
Japanese  students  to  practice  English  language  skills  and  exchange  cultural  information.  The  purpose  of  this  pilot 
study  was  to  understand  the  Taiwanese  EEL  learners’  perceptions  of  practice  English  language  skills  through 
distance  technologies  and  to  identify  difficulties  or  facilitators  in  turn -taking  and  online  intercultural  communication 
during  desktop  videoconference  (DVC).  This  paper  reports  primary  findings  in  the  pilot  study  carried  out  during  the 
school  year  of  2001-2002  and  discusses  possible  improvement  for  future  connections. 

Rationale 

Distance  technologies  have  commonly  been  used  for  delivery  of  instruction  in  the  West  (e.g.,  the  United 
States,  Great  Britain,  and  other  English-speaking  countries)  in  recent  decades.  Historically,  distance-learning 
technologies  have  included  ordinary  mail,  broadcast  TV,  and  radio  (Bates,  1995).  The  use  of  telecommunications  by 
the  general  public  started  in  1991  when  the  Internet  evolved  from  a tool  used  primarily  by  the  military  and  scientists 
to  a tool  used  by  educators  (Ryder  & Hughes,  1997).  In  highly  developed  countries  such  as  the  United  States, 
Australia,  and  Great  Britain,  courses  in  open  universities  and  higher  education  institutions  are  now  designed  to  be 
delivered  fully  or  partially  through  distance  technologies  (Simich-Dudgeon,  1998). 

Taiwanese  educators  regard  the  West  as  a provider  of  models  for  development  and  therefore,  they  follow 
Western  scientific  progress.  When  telecommunications  started  to  be  used  in  education  in  the  western  part  of  the 
world,  Taiwan  was  not  far  behind.  Since  the  mid-1990s,  the  Internet  has  become  a widely  adopted  communication 
tool  in  Taiwan.  Over  time,  the  Internet  infrastructure  in  Taiwan  is  also  being  improved;  therefore,  the  government 
encourages  researchers  and  scholars  to  study  and  explore  in  the  field  of  telecommunications  for  teaching  and 
learning. 

In  foreign  language  learning,  researchers  such  as  Cononelos  and  Oliva  (1993),  Davis  and  Chang  (1994/95), 
and  Leh  (1997)  suggested  using  telecommunications  technologies  to  create  authentic  language-learning  situations 
for  learners  to  communicate  with  foreign  speakers  directly  and  to  develop  intercultural  competence.  In  Taiwan, 
when  such  a method  was  implemented  for  English  language  learning  and  communication,  it  remedied  Taiwan’s 
geographical  isolation  as  an  island  and  provided  opportunities  for  EEL  learners  to  learn  and  converse  in  an  English 
communication  environment. 

The  potential  for  conducting  second  or  foreign-language  instruction  via  telecommunications  has  not  been 
ignored  by  educators  worldwide.  Since  late  1980s  and  early  1990s,  telecommunications  projects  with  language 
learning  have  been  conducted  to  teach  languages,  such  as  EEL  (Cifuentes  & Shih,  1999;  Davis  & Chang,  1994/95; 
Kamhi-Stein,  1997;  Soh  & Soon,  1991;  Sullivan,  1993);  French  (Oliver  & Nelson,  1997);  Spanish  (Leh,  1997); 
Portuguese  (Kelm,  1992);  and  Russian  (Samsonov,  1998). 

The  conclusions  of  these  projects  mostly  stated  the  benefits  and  limitations  of  these  online  connections. 
Benefits  included  accessibility  to  people,  language  improvement,  acquisition  of  cultural  knowledge,  and  the 
development  of  communicative  and  critical  thinking  skills.  On  the  other  hand,  limitations  included  technical 
difficulties  and  the  lack  of  response  from  students. 
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With  the  synchronous  connections  via  videoconferencing  in  the  second  or  foreign  language  teaching,  not 
many  teachers  adopted  this  method  in  their  classrooms  due  to  the  high  cost  of  the  equipment  and  poor  quality  of  the 
results  (O’Dowd,  2000a).  Among  the  ones  that  tried  large-group  videoconferencing  or  desktop  videoconferencing  in 
language  teaching  or  learning  situations  are  University  of  Leon  in  Spain  (O’Dowd,  2000b),  University  of  Exeter  and 
University  College  London  (Buckett  & Stringer,  1997;  Bucket,  Stringer,  & Datta,  1999;  Stringer  & Buckett,  1999), 
three  primary  schools  in  North  Yorkshire  (Bell  & Maynard,  1996),  and  London  School  of  English  and  Eoreign 
Languages  (Goodfellow,  Jefferys,  Miles,  and  Shirra,  1996).  These  papers  delineated  the  experiences,  achievements, 
and  limitations  of  language  teaching  and  learning  via  videoconferencing. 

In  Taiwan,  there  have  been  studies  on  EEL  teaching  and  learning  through  telecommunications  such  as 
electronic  mail  by  Hsien-Chin  Liou  (1996)  and  Meei-Ling  Liaw  (1997).  In  her  study,  Liou  studied  syntactic  errors, 
topics  discussed,  and  communicative  functions  in  the  e-mail  writings  by  Chinese  EEL  college  students  who 
corresponded  with  foreign  pen  pals.  She  further  developed  a Computer-Mediated  Communications  model  to 
facilitate  EEL  learning  (Liou,  1998).  Liaw  found  multiple  advantages  through  effective  integration  of  e-mail  writing 
between  two  classes  of  EEL  students  in  Taiwan. 

Hardly  any  English  courses  in  Taiwan  are  taught  through  the  synchronous  format  (Liu,  2001),  let  alone 
having  students  communicate  with  teachers  or  EEL  learners  of  other  countries  via  videoconferencing  systems. 
Nevertheless,  a few  researchers  in  other  areas  have  started  to  use  large-group  and  desktop  videoconferencing  in  their 
courses.  Shih-Chang  Hsin  (1997)  had  graduate  students  in  the  Institute  of  Teaching  Chinese  as  a Eoreign  Language 
at  National  Taiwan  Normal  University  tutoring  students  in  University  of  Hawaii  via  desktop  videoconferencing.  In 
his  second  study  (Hsin,  1999),  he  compared  the  use  of  large-group  videoconference  against  desktop 
videoconference.  Chen,  Chen,  Guo,  Lin,  Chen,  Lee,  and  Chen  (1998)  studied  the  effectiveness  of  teaching  a general 
medical  education  course  “health  promotion”  via  videoconference  system  in  the  National  Taiwan  University  College 
of  Medicine.  The  results  showed  that  students’  overall  satisfactory  rate  on  the  distance  delivery  mode  of  the  course 
was  60%. 

Methodology 

Procedures 

Eorty-four  Taiwanese  EEL  students  taking  the  course  English  Language  Learning  through  Multimedia  I and 
//and  30  Japanese  EEL  students  participated  in  this  project  in  the  fall  semester  of  2001  and  spring  semester  of  2002. 
Eirst,  as  the  semester  started,  participants’  digital  pictures  and  self-introductions  were  uploaded  on  to  the  project 
website  (these  were  situated  under  the  course  website:  http://night.ntcc.fju.edu.tw/doris/media/media.htnj).  Second, 
participants  also  received  instructions  on  the  basics  of  telecommunication  technologies  and  the  use  of  DVC  software 
iVisit  v2.6  b7.  In  addition,  they  watched  a section  of  the  videotape  entitled  Learn  & Live  produced  by  The  George 
Lucas  Educational  Eoundation.  This  prepared  them  for  situations  of  DVC  as  most  of  them  were  unfamiliar  with 
DVC. 

During  the  semester,  students  of  both  ends  carried  out  web-board  discussions  and  multiple  DVC 
connections.  Eive  web-board  discussion  areas  were  set  up  using  Beseen,  a Looksmart  service.  With  DVC,  the 
Taiwanese  students  were  divided  into  groups  of  three  or  four  people  for  separate  connections.  Each  student  took 
turns  to  speak  to  the  Japanese  students.  An  assistant  was  present  to  help  with  the  technical  setup,  which  allowed  the 
researcher  to  take  observation  notes.  Topics  discussed  extended  beyond  the  students’  learning  of  EEL.  The 
coordinators  at  both  ends  provided  discussion  starters  on  topics  such  as  digital  divide,  eWorld,  wired  schools,  and 
job  and  technology.  The  students  were  asked  to  reflect  on  their  learning  and  experiences  after  the  connections. 

Method 

This  study  employs  a qualitative  research  design.  This  design  enables  the  researcher  to  inquire, 
comprehend,  and  describe  the  experiencing  world  of  the  participants  and  the  meaning  of  these  experiences  (Bodgan 
& Biklen,  1998;  Lincoln  & Cuba,  1985). 

The  data  sources  included  desktop  videoconferencing  records  (i.e.,  chat  files),  observation  notes,  reflective 
papers,  and  interview  transcripts  of  participants.  Each  time  data  are  gathered,  information  has  been  analyzed  using 
content  analysis  procedures.  In  the  paper,  Taiwanese  students  are  identified  as  TI-T44  and  Japanese  students  as  JI- 
J30. 

Results  and  Discussion 

A Web-board  was  set  up  for  participants  to  communicate  with  each  other  before  and  after  the  DVC  sessions. 
They  each  sent  a self-introduction  on  to  the  board.  Eour  students  posed  technical  questions  regarding  the  use  of  the 
web-board.  Teachers  in  Taiwan  and  Japan  took  turns  to  pose  discussion  starters  each  week  but  not  many  students 
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responded  to  the  questions.  Only  three  Taiwanese  students  and  two  Japanese  students  wrote  long  responses.  There 
are  two  reasons  for  this  phenomenon.  One  reason  is  that  the  participants  were  not  forced  to  use  the  discussion  board. 
Another  reason  is  that  Asian  students  dislike  expressing  opinions  in  the  public  (Hofstede,  1986;  Shih,  2000).  T7 
expressed  that  she  felt  very  shy  to  interact  with  the  instructor  in  the  class,  not  even  to  mention  having  interactions 
with  unfamiliar  Japanese  students  in  the  discussion  board. 

With  the  DVC  connections,  each  group  of  students  was  given  a worksheet  for  the  ice-break  session.  The 
questions  on  the  worksheet  were  related  to  personal  background  and  individual  uses  of  the  Internet.  All  groups 
completed  the  worksheet  successfully.  In  the  next  two  sessions,  the  participants  talked  about  the  cultures  of  Taiwan 
and  Japan  and  their  EFL  learning  experiences. 

Data  showed  the  most  positive  outcome  of  the  connection  was  the  pleasure  that  the  participants  exhibited 
regarding  learning  about  the  cultures  of  Taiwan  and  Japan.  Forty-four  out  of  44  Taiwanese  students  expressed  in 
their  written  reflections  that  they  enjoyed  or  liked  this  activity  very  much.  Other  benefits  included  opportunity  to 
experience  the  use  of  this  “fun  hi-tech”  (T22),  authentic  language  practice,  and  making  foreign  friends.  Many  noted 
that  they  learned  what  distance  learning  is  and  became  un-afraid  of  the  machines.  Several  said  they  realized  the 
importance  of  social  presence.  “If  I want  to  see  the  other’s  face,  I need  the  web-cams.  It’s  a useful  tool  to 
communicate  to  others”  (T36).  Thus  they  wanted  to  buy  web-cams  after  the  class  and  would  definitely  use  DVC  in 
their  own  time  by  connecting  to  friends  or  chat  with  foreigners  on  the  Internet. 

Difficulties  encountered  were  interrupted  audio  transfer  in  DVC,  confused  turn -takings  when  there  were 
more  than  two  communicators,  and  misinterpreted  underlying  messages.  The  Internet  speed  was  slow  during  several 
DVC  sessions.  Once  the  audio  transfer  became  problematic,  the  participants  immediately  switched  to  use  the  chat 
function  in  the  software.  Every  student  in  Japan  had  access  to  a computer;  therefore,  at  times  there  were  more  than 
one  Japanese  student  connected  to  talk  to  one  Taiwanese  student.  Data  showed  that  the  one-on-one  connection 
allowed  for  a more  smooth  communication  (see  chat  transcript  1;  the  names  are  pseudonyms). 

Chat  transcript  1: 

Multimedia65,  Fu-Jen:  yes,  do  you  like  music? 

RR:  you  can  call  me  Ricky 

Multimedi  a65,  Fu-Jen:  you  can  call  me  Austin. 

RR:  yeah...  I like  music 
RR:  austin  powers 

Multimedia65,  Fu-Jen:  what  type  of  music  do  you  like? 

RR:  I like  Brit -Pop.,  and  hip  hop  and  soul  music 

When  there  were  multiple  communicators  in  DVC,  multiple  conversations  went  on  at  the  same  time  and 
caused  confusion  for  some  participants.  One  person  decided  to  disconnect  herself  halfway  during  the  connection 
(see  chat  transcript  2). 

Chat  transcript  2 

Multimedia64,  Fu  -Jen:  what's  the  name  of  your  uni 
JJ:  uni? 

Multimedia64,  Fli-Jen:  University 
JJ:  ferns 

English,  Fu-Jen:  because  she  want  to  tell  you  something 

MM:  Or,  if  that  doesn’t  works 

English,  Fu-Jen:  she  will  go  to  Japan 

MM:  please  send  to  mm83@hotmail.com 

MM:  OH! ! ! please  send  me  email 

MM:  then  I can  tell  you  many  thing  1 ! 

English,  Fu-Jen:  Her  : p@yahoo.com.tw 
Multimedia66,  Fu-Jen:  WW  is  working  in  a hotel 
Multimedia66,  Fu-Jen:  you  are  thinking  too  much... 

Multimedia66,  Fu-Jen:  it  too  difficult  to  us 
PP:  Question  F 

Multimedia64,  Fu-Jen:  Where  is  JJ? 

Multimedia64,  Fu-Jen:  Is  JJ  here? 

PP:  Nope!!! 
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A system  must  be  developed  to  help  the  participants,  especially  EFL  learners,  to  communicate  functionally  on  turn- 
takings  in  the  DVC  and  chat  situations.  Murphy  & Collins  (1998)  identified  the  need  to  train  students  to  use 
communication  conventions  and  protocols  for  clear  communication. 

Educational  Significance 

This  project  is  significant  in  the  following  points.  First,  it  provides  students  who  study  in  the  humanity 
areas  (such  as  the  English  majors)  experiences  of  using  online  technologies  for  learning  and  authentic 
communication  with  foreigners  in  English.  Many  students  of  English-language  majors  do  not  have  much  access  to 
technological  courses  or  are  fearful  of  technologies.  Through  such  experiences,  they  will  understand  multiple  ways 
of  using  technologies  to  learn  English  as  a foreign  language  and  start  to  apply  the  skills  into  their  foreign  language 
learning  situations.  Second,  the  study  provides  insights  for  distance  educators,  both  for  those  in  Taiwan  and  for 
those  in  other  countries  who  have  Taiwanese  students  enrolled  in  courses  that  are  delivered  via  telecommunications. 
The  results  of  the  study  will  help  these  instructors  to  design  curriculum  by  selecting  and  incorporating  appropriate 
asynchronous  and  synchronous  technologies . Third,  the  online  intercultural  communications  issues  experienced  in 
this  project  assist  telecommunications  users  with  more  effective  communication.  It  helps  users  become  aware  of  and 
anticipate  problems  when  coming  into  contact  with  people  of  other  cultures  via  distance  technologies. 

The  Future 

During  the  school  year  of  2002-2003,  this  project  will  be  funded  by  National  Science  Council  of  the 
Republic  of  China.  The  researcher  proposes  to  make  the  following  adjustment.  First,  participants  will  be  reduced  to 
30  students  in  Taiwan  to  make  the  number  equal  to  that  in  Japan.  This  allows  the  researcher  to  carry  out  many  more 
in-depth  interviews  with  the  Taiwanese  participants.  Second,  the  processes  of  connection  and  the  interviews  will  be 
recorded  through  a video  camera,  as  DuFon  (2002)  noted  that  good  video  footages  provide  valuable  and  usable  data, 
and  give  more  linguistic  information  than  field  note  taking.  Third,  in  addition  to  showing  clips  from  the  videotape 
Learn  and  Live,  another  video  entitled  New  Business  of  Paradigms  can  be  shown  to  prepare  participants  for  change 
of  a new  paradigm.  Fourth,  participants  will  be  given  articles  related  to  intercultural  communication,  such  as 
“Cultural  differences  in  Teaching  and  Learning’’  by  Geert  Hofstede  (1986)  and  “Intercultural  Communication  via  E- 
mail  Debate’’  by  G.  M.  Chen  (1998).  The  readings  will  prepare  participants  to  self  analyze  intercultural  problems 
evolved  in  the  telecommunications  setting. 
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Abstract 

This  study  investigated  the  use  of  inquiry-based  multimedia  learning  environment  as  a way  to  increase 
students  ’ problem-solving  skills,  especially  female  students  in  a ninth-grade  astronomy  course.  Using  a pre-  and 
posttest  design,  the  study  used  multiple  choice  and  written  essay  questions  to  measure  students’  content 
understanding  and  problem-solving  skills.  The  results  ofANOVA  conclude  that  inquiry-based  multimedia  learning 
is  viable  and  effective  for  improving  content  understanding  and  problem-solving  skills  with  all  students,  especially 
females. 

Introduction 

The  investigation  of  gender  differences  has  been  a significant  area  of  study  for  many  researchers  in  science 
(American  Association  of  University  Women  [AAUW],  1992;  Craig,  1999;  Mael,  1998;  Scanlon,  2000).  After 
years  of  research  outlining  inequities,  there  still  remains  a discrepancy  between  the  performance  of  females  and 
males  in  science  courses  (Shymansky,  Hedges,  & Woodworth,  1990;  Third  International  Mathematics  and  Science 
Study,  1998).  More  efforts  are  needed  to  remove  the  obstacles  to  gender  equity  (Kirkpatrick  & Cuban,  1998; 
Littleton,  Light,  Joiner,  Messer,  & Barnes,  1992;  Whitelock  et  ak,  1995).  Researchers  indicate  that  females  often 
become  uncomfortable  and  uninterested  in  science  early  in  the  educational  process  (Jewett,  1996;  Valentine,  1998). 
AAUW  (1992)  and  National  Assessments  of  Educational  Progress  (1979)  report  that  females  have  lower 
achievement  scores  in  science  knowledge,  hold  a less  positive  attitude  toward  science,  and  participate  in  far  fewer 
science-related  activities  than  do  males  (Schibeci  & Riley,  1986).  Hill,  Pettus,  and  Hedin  (1990)  cite  that  females  in 
science-related  fields  exhibit  a high  level  of  science  anxiety. 

A variety  of  factors  contribute  to  the  poor  performance  of  female  students  in  science.  These  factors  include 
social  expectations  related  to  gender-role  in  career  choice,  teacher  bias,  and  discrepancy  in  the  amount  of  hands-on 
experiences  (Jewett,  1996).  Researchers  believe  that  a primary  reason  for  gender  differences  is  that  females  are  not 
given  the  same  opportunities  to  learn  as  male  students  (Valentine,  1998).  Females  receive  an  inferior  education  to 
males  in  the  use  of  science  equipment  and  materials.  In  a science  laboratory,  for  example,  male  students  more  often 
work  with  the  equipment  while  female  students  write  down  the  observations  (Rosser,  1990).  As  a result,  females  do 
not  gain  experience  or  confidence  in  the  manipulation  of  science  equipment  (Hill,  Pettus,  & Hedin,  1990;  Kahle, 
1996;  Rosser,  1990;  Sjoberg,  1993). 

This  lack  of  experiences  lowers  females’  self  confidence  in  their  science  abilities  and  fosters  negative 
attitudes  toward  science  (Kahle,  1996).  It  might  account  for  a large  portion  of  the  differences  between  males  and 
females  in  science  achievement  (Jewett,  1996;  Kahle,  1996).  Researchers  suggest  that  females’  performance  in 
science  will  improve  by  using  instructional  methods  such  as  inquiry -based  approaches  (Greenberg -Lake,  1991;  Hill 
et  ak,  1990;  Lock,  1992;  McLaren  & Gaskell,  1995;  Shakeshaft,  1995).  These  methods  provide  a widervariety  of 
science  experience  for  all  students,  particularly  females.  These  opportunities  include  practice  observing,  measuring, 
and  testing  hypotheses  (Eccles,  1989;  Freedman,  2001;  Matyas,  1985;  Shakeshaft,  1995).  Inquiry -based  approaches 
are  a key  to  keeping  females  interested  in  science  and  participating  in  hands-on  experience  (Freedman,  2001; 

Jenkins  & MacMonald,  1989;  Matyas,  1985). 

Inquiry  Learning 

Inquiry -based  approaches  have  been  described  as  a “pedagogical  method  combining  intellectual 
questioning  with  student-centered  discussion  and  discovery  of  pivotal  concepts  through  laboratory  activities’’ 
(National  Research  Council,  1996).  Inquiry  emphasizes  observing,  classifying,  formulating  hypotheses,  identifying 
and  controlling  variables,  experimenting,  and  making  valid  conclusions  (Gagne,  1970).  Inquiry  has  been  shown  to 
be  a successful  intervention  technique  in  science  (Anderson,  1993;  Doran,  Boorman,  Chan,  & Hejaily,  1993;  Lock, 
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1992;  Sjoberg,  1993;  Thomas,  1986).  Inquiry -based  activities  facilitate  scientific  problem  solving  because  they  let 
students  question  their  own  observations,  generate  and  refine  hypotheses,  deal  with  experimental  data,  test  those 
hypotheses  by  experimentation,  and  evaluate  evidence  (Bybee,  1993;  Collins,  1998;  Kuhn,  Amsel,  & O'Loughlin, 
1988;  Moore,  1993;  National  Research  Council,  1996;  Uno,  1990;  Windschitl,  2000). 

Many  studies  have  shown  that  inquiry -based  approaches  produce  significantly  greater  science  process  skills 
than  conventional  approaches  (Chiappetta  & Russell,  1982;  Glasson,  1989;  Krajcik  et  al,  1998).  Moreover,  some 
groups  report  that  students’  cognitive  ability,  science  achievement,  and  scientific  problem  solving  improve  if 
students  experience  inquiry -based  approaches  (Bredderman,  1983;  Greenberg -Lake,  1991;  Haury,  1993;  Hill  et  al., 
1990;  Lindberg,  1990;  Lock,  1992;  McLaren  & Gaskell,  1995). 

In  addition,  research  has  shown  that  inquiry -based,  hands-on  strategies  increase  the  science  success  of 
female  students  (Bredderman,  1982;  Shymansky  et  al.,  1990).  The  inquiry  activities  enable  females  to  participate  in 
hand-on  experiences,  to  see  relevance  in  the  application  of  science,  and  to  make  science  important  for  them  to  learn 
(Anderson,  1993;  Greenberg -Lake,  1991;  Hill  et  al.,  1990;  Lock,  1992;  McLaren  & Gaskell,  1995;  Thomas,  1986). 
Researchers  have  demonstrated  that  increasing  these  experiences  for  female  students  boosts  their  attitudes  toward 
science  and  their  achievement  (Shakeshaft,  1995;  Simpson  & Oliver,  1985).  For  instance,  Eccles  (1989)  reports  that 
females  who  have  more  inquiry -based  experiences  and  opportunities  can  develop  greater  confidence  in  their 
abilities,  more  interest  in  subjects,  and  more  interest  in  pursuing  careers  in  mathematics  and  science-related  fields. 

Inquiry -based  approaches  have  been  applied  in  many  innovative  ways  to  assist  science  instruction.  Most 
notably  computer  technology  lets  students  experience  authentic  problem-solving  activities  that  are  difficult  or 
impossible  to  create  in  real  classroom  situations  (Geban,  Askar,  & Ozkan,  1992;  Windschitl,  2000;  Yager,  1995; 
Zietsman,  1986).  Inquiry -based  approaches  that  incorporate  new  technologies  allow  students  to  experience 
authentic  science,  and  they  result  in  improved  science  achievement  (Bybee,  1993;  Collins,  1998;  Kuhn  et  al.,  1988; 
Kuhn.D.,  Amsel,  & O'Loughlin,  1988;  Mayer,  Moreno,  Boire,  & Vagge,  1999;  Moore,  1993;  National  Research 
Council,  1996;  Pea,  1993;  Uno,  1990;  Windschitl,  2000). 

Purpose  of  the  Study 

Based  on  prior  research  findings,  inquiry  learning  incorporating  technology  improves  students’ 
achievement,  especially  females  (Greenberg -Lake,  1991;  Hill  et  al.,  1990;  Jenkins  & MacMonald,  1989;  Lock, 

1992;  McLaren  & Gaskell,  1995;  Shakeshaft,  1995;  Simpson  & Oliver,  1985).  This  study  investigates  two  specific 
research  questions: 

■ Does  a multimedia  program  designed  around  inquiry -based  approaches  improve  students’  conceptual 
understanding  and  problem-solving  skills? 

■ Does  a multimedia  program  designed  around  inquiry -based  approaches  have  a differential  effect  on  the 
conceptual  understanding  and  problem-solving  skills  of  male  and  female  students? 

Instructional  Context 

This  study  was  conducted  within  the  context  of  Astronomy  Village®:  Investigating  the  Universe™  (see 
http://www.cotf.edu/AV/avl  .html).  Astronomy  Village  uses  the  metaphor  of  living  and  working  at  a mountaintop 
observatory  (the  village)  as  the  primary  interface  from  which  students  investigate  contemporary  problems  in 
astronomy  (Pompea  & Blurton,  1995  Jan/Feb).  The  environment  includes  10  investigations  covering  a broad  cross- 
section  of  current  research  in  astronomy.  In  Astronomy  Village  a student  research  team  is  aided  by  a virtual  mentor 
who  immerses  team  members  in  science  concepts  and  science  inquiry  skills.  The  research  path  diagram  provides  an 
overall  guideline  for  the  use  of  the  program  (see  figure  1).  Student  teams  proceed  along  a path  of  activities 
represented  by  various  icons  in  the  research  path  diagram.  Each  icon  guides  them  to  the  village’s  virtual  research 
facilities.  At  each  facility  the  students  gain  information  relative  to  their  investigation. 

The  design  principles  of  Astronomy  Village  come  from  inquiry  processes  emphasizing  observing, 
classifying,  experimenting,  and  making  valid  conclusions  (Gagne,  1970).  Eor  example.  Astronomy  Village  used  four 
phases  to  support  the  inquiry  process:  (1)  identifying  questions  to  investigate  in  the  background  research  phase,  (2) 
designing  investigations,  (3)  conducting  investigations  in  the  data  collection,  data  analysis,  and  data  interpretation 
phases,  and  (4)  formulating  and  communicating  conclusions  in  the  presentation  phase  (Howard,  McGee,  Shin,  & 
Regina,  2001;  Robitaille  et  al.,  1993).  Within  each  phase  students  may  complete  up  to  six  activities,  such  as 
simulations,  hands-on  experiments,  thought  questions,  LogBook  entries,  and  library  research. 

In  the  phase  of  identifying  questions  to  investigate,  students  are  presented  with  contemporary  issues  in 
astronomy.  The  questions  are  currently  important  to  the  scientific  community  and  not  fully  answered  by  scientists. 

In  the  Nearby  Stars  investigation,  for  example,  students  are  asked  to  measure  the  distance  to  nearby  stars,  using  a 
technique  based  on  the  parallax  angle. 
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FIGURE  1.  The  Astronomy  Village  interface  and  the  research  path  diagram  for  the  Nearby  Stars  investigation. 
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The  background  research  phase  encourages  students  to  examine  articles  and  other  sources  of  information  to 
see  what  is  already  known  in  light  of  experimental  evidence.  The  activities  in  this  phase  include  reading  articles  in 
the  virtual  library  and  listening  to  lectures  in  the  virtual  theater.  The  library  contains  a large  number  of  resources 
directly  or  indirectly  relevant  to  the  investigation. 

As  part  of  the  designing  investigations  phase,  investigations  were  developed  to  allow  students  to  use 
multiple  methods  to  address  the  problem.  The  virtual  environment  provides  an  authentic  research  context  based  on 
Kit  Peak  Observatory  in  Arizona. 

In  the  conducting  investigations  phase  the  environment  provides  scaffolding  to  help  students  successfully 
complete  the  investigation.  The  students’  virtual  mentors  send  e-mail  messages  providing  scaffolding  on  how  to 
analyze  the  image  data  using  an  image-processing  software  program.  This  gives  students  an  expert’s  suggestions 
and  encouragement  for  pursuing  their  investigation.  The  software  contains  all  the  resources  s tudents  need  to 
conduct  the  investigation,  including  articles,  data  analysis  tools,  and  organizational  help.  Moreover,  students  are 
given  choices  in  selecting  activities  most  beneficial  for  solving  the  problem. 

In  Nearby  Stars,  for  instance,  students  collect  from  the  virtual  observatory  images  showing  stellar 
movements.  In  the  computer  lab  they  use  image-processing  techniques  to  analyze  these  images  taken  at  different 
time  periods.  Stars  that  move  during  a six-month  period  are  nearby  stars.  Using  the  movement,  students  calculate 
the  parallax  angle  of  the  nearby  star.  That  angle  then  lets  them  calculate  the  star’s  distance  from  Earth.  Students  see 
how  they  can  measure  the  distance  to  nearby  stars.  They  then  propose  answers,  explanations,  and  predictions. 

In  the  phase  of  formulating  and  communicating  conclusions,  students  have  opportunities  to  develop 
formative  conclusions  and  receive  feedback  from  their  peers  and  the  teacher.  The  LogBook  provides  a place  for 
students  to  jot  down  their  observations,  formulate  written  conclusions,  and  draw  diagrams  about  their  investigation. 
Students  are  encouraged  to  develop  verbal  conclusions  as  well  as  draw  diagrams  within  their  LogBook.  At  the  end 
of  the  investigation,  students  host  a virtual  press  conference  in  which  they  click  on  virtual  reporters  who  ask  them 
questions  about  the  investigation.  Students  respond  in  written  form  to  the  questions  and  store  the  responses  in  their 
LogBook.  Finally,  students  present  their  findings  to  their  classmates  in  an  oral  presentation.  It  lets  them  articulate 
their  understanding  by  discussing  their  results  with  their  team  members  as  well  as  the  entire  class.  It  also  supports  a 
social  environment  that  allows  students  to  get  feedback  by  comparing  their  works  to  others. 

Many  aspects  oi  Astronomy  Village  can  be  identified  as  potentially  appealing  to  females’  interests, 
therefore  their  achievement.  For  example,  the  program  deals  with  investigation  questions  based  on  contemporary 
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issues.  The  real-life  contexts  enable  all  students,  especially  females,  to  support  social  interaction  and  to  see 
relevance  in  the  application  of  science  (Anderson,  1993;  Freedman,  2001).  Additionally,  the  main  techniques  of 
Astronomy  Village  emphasize  observing,  measuring,  and  hypothesis  testing,  which  female  students  lack  exposure  to 
and  experiences  in  the  current  science  classrooms  (Jenkins  & McDonald,  1989).  Increasing  these  experiences  has 
been  demonstrated  as  a successful  intervention  technique  for  all  students,  especially  females  (Simpson  & Oliver, 
1990;  Shakeshaft,  1995).  Finally,  students  in  Astronomy  Village  conduct  their  own  experiments  and  activities  using 
provided  information  and  virtual  mentor’s  messages.  The  environment  provides  students  pupil-centered 
experiments,  which  is  a preferred  teaching  style  by  females  (Freedman,  2001). 

Method 

Subjects 

The  participants  in  this  study  were  122  ninth-grade  students  attending  a high  school  in  a small  working- 
class  community  near  a large,  Midwestern  city.  All  ninth-grade  students  enrolled  in  the  Earth  and  space  science 
course  were  invited  to  participate.  Ninth-graders  were  chosen  because  Astronomy  Village  was  designed  to  support 
ninth-grade  science  classes.  The  school  n&td  Astronomy  Village  in  regular  science  classes  consisting  of  mixed  high- 
and  low-ability  students. 

Of  the  122  students  who  completed  surveys,  103  (84  percent)  students  were  Caucasian  (white).  One  male 
student  (1  percent)  was  black.  One  male  student  (1  percent)  was  Asian-American.  Two  male  students  (2  percent) 
were  Hispanic.  Five  females  and  two  males  (6  percent)  were  other  races.  Eight  students  (7  percent)  did  not  indicate 
their  race.  The  gender  breakdowns  of  the  sample  were  62  females  (5 1 percent)  and  57  males  (47  percent).  Three 
students  (2  percent)  did  not  indicate  their  gender. 

Instruments 

In  this  study  instruments  were  developed  to  measure  the  students’  problem-solving  skills  and  content 
understanding,  using  multiple  choice  and  written  essay  questions.  The  test  items  were  subjected  to  an  extensive 
construct  validation  process  (Hong,  1998;  Shin,  Jonassen,  & McGee,  in  press).  Subject  matter  experts,  curriculum 
developers,  educational  researchers,  and  teachers  iteratively  reviewed  the  test  items  with  regard  to  content  accuracy, 
readability,  vocabulary  level,  and  appropriateness  for  ninth-grade  students.  In  addition,  the  test  items  were  pilot 
tested  with  students  using  think-aloud  protocols. 

As  a measure  of  content  understanding,  students  were  asked  to  classify  important  concepts  relating  to  a 
research  study  by  selecting  relevant  concepts  from  a list  (Clark,  1990;  Hong,  1998).  Students  were  required  to 
analyze  the  problem  statement  to  determine  which  concepts  are  relevant  to  the  solution.  Additionally,  to  gauge  how 
the  concepts  are  organized  in  a meaningful  way,  students  were  asked  to  describe  the  relationships  between  the 
chosen  concepts  (Jonassen,  Beissner,  & Yacci,  1993). 

Two  problems  are  developed  to  measure  students’  problem-solving  skills  in  astronomy.  On  the  assessment 
students  are  required  to  present  and  explain  their  approach  for  solving  given  open-ended  questions  (Hong,  1998). 

One  of  the  problems  focuses  on  extensions  to  the  Nearby  Stars  investigation;  the  other  focuses  on  extensions  to  the 
Variable  Stars  investigation.  These  represent  2 of  10  research  investigations  in  Astronomy  Village.  In  the  Nearby 
Stars  investigation  students  use  the  concept  of  parallax  to  measure  the  distance  to  nearby  stars.  In  the  Variable  Stars 
investigation  they  use  the  inverse  square  law  to  determine  the  distance  to  a nearby  galaxy. 

The  scoring  rubric  was  developed  similar  to  other  rubric  systems  (Baxter,  Glaser,  & Raghavan,  1993).  It 
was  reviewed  by  content  experts,  educational  researchers,  curriculum  developers,  and  teachers  to  make  sure  it  was 
developed  appropriately  and  includes  all  important  aspects  for  measuring  students’  problem-solving  skills  in 
astronomy.  The  scoring  rubric  was  tested  for  construct  validity  using  the  method  of  instructional  sensitivity,  which 
compares  differences  in  the  response  of  ejqterts  and  novices  (Gall,  Borg,  & Gall,  1996).  The  result  of  construct 
validation  indicated  that  the  scoring  rubric  can  discriminate  differences  between  students  who  demonstrate  a well- 
organized  problem-solving  process  and  those  who  demonstrate  a disorganized  process  in  given  problem  situations  (p 
< .000)  (Hong,  1998;  Shin  et  ak,  in  press).  In  addition,  the  instrument  has  been  shown  to  correlate  with  measures  of 
metacognition,  science-related  attitudes,  and  motivation  (Hong,  1998).  Three  raters  scored  the  students’  responses 
individually.  The  average  overall  interrater  reliability  was  .82. 
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Procedure 

The  study  was  conducted  during  a two -month  period.  The  teacher  and  students  were  asked  to  provide 
demographic  information  and  complete  a release  form  before  any  data  was  collected.  Before  students  used 
Astronomy  Village,  the  teacher  and  an  investigator  spent  two  classroom  periods  administering  the  pretest. 

During  science  classes  the  research  team  worked  closely  with  school  district  members  and  teacheis  to 
support  the  effective  implementation  of  Astronomy  Village.  The  research  team  provided  all  the  technical  support 
through  the  entire  program.  One  team  member  assisted  teachers  in  designing  classroom  activities,  including  lesson 
plans.  Another  team  member  monitored  the  classroom  activities  to  determine  if  the  program  was  being 
implemented  properly.  This  team  member  also  helped  to  resolve  any  technical  issues  students  experienced  in  the 
implementation. 

All  students  used  a PC  computer  and  Astronomy  Village.  They  worked  in  55  -minute  class  periods,  five 
times  per  a week,  for  a total  of  15  class  periods.  The  teacher  taught  five  classes.  Each  class  had  23-26  students. 

The  teacher  had  previous  experience  itaching  Astronomy  Village. 

The  teacher  randomly  assigned  students  to  conduct  one  of  two  Astronomy  Village  investigations.  Nearby 
Stars  or  Variable  Stars.  Each  investigation  took  students  through  five  phases  of  research:  background  research,  data 
collection,  data  analysis,  data  interpretation,  and  presentation  of  results.  During  the  investigation  students  used  an 
electronic  LogBook  to  record  their  scientific  notes  and  observations.  At  the  end  of  the  program,  each  student  team 
presented  its  results  to  the  class. 

After  students  iini&hed  Astronomy  Village,  a posttest  was  administered  during  two  classroom  periods.  In 
both  the  pre-  and  posttests,  each  student  took  all  the  tests.  These  included  conceptual  understanding,  the  Nearby 
Stars  problem,  and  the  Variable  Stars  problem.  In  the  tests  the  problems  were  printed  individually  for  each  student. 
After  the  tests  were  distributed,  the  students  read  the  introduction  before  being  told  to  solve  the  problems.  Students 
were  not  allowed  to  use  computers  or  any  resources  while  completing  the  tests.  There  was  no  time  limitation  to 
finish  all  the  instruments. 

Data  Analysis 

The  research  sought  to  explore  how  the  inquiry -based  multimedia  program  would  affect  students’, 
especially  females’,  problem-solving  skills  in  science.  Dependent  variables  included  content  understanding  and 
problem  solving.  The  data  was  analyzed  separately  using  analysis  of  variance  (ANOVA).  In  the  study  students 
conducted  only  one  investigation  (either  Nearby  Stars  or  Variable  Stars),  and  they  took  tests  in  conceptual 
understanding  as  well  as  Nearby  Stars  and  Variable  Stars  problem  solving.  Thus,  the  scores  on  the  test  that  is  based 
on  the  path  that  students  did  not  conduct  serves  as  a control  group.  The  analysis  for  conceptual  understanding  and 
problem  solving  was  a 2 (treatment:  experimental  vs.  control)  x 2 (test  occasion:  pretest  vs.  posttest)  analysis  of 
variance  in  which  the  treatment  was  between  subjects  variables  and  the  test  occasion  was  a within-subjects  variable. 
Within  the  treatment  group,  2 (test  occasion:  pretest  vs.  posttest)  x 2 (gender:  females  vs.  males)  analysis  of 
variance  was  conducted  as  well. 

Results 

Conceptual  Understanding 

Table  1 shows  the  means  and  mean  proportions  of  problems  answered  correctly  on  the  pretest  and  posttest 
and  the  standard  deviations  for  these  means.  It  also  provides  descriptive  statistics  of  two  dependent  variables  based 
on  gender.  Using  scores  of  experimental  and  control  groups  as  the  between-group  factor  and  the  pretest  and  posttest 
as  the  within-subject  factor,  a two-way  ANOVA  was  employed  to  analyze  the  differences  between  students’ 
improvement  on  conceptual  understanding.  Examination  of  the  results  for  the  significant  interaction  revealed  that 
disordinal  interaction  existed  between  the  two  independent  variables,  test  occasion  and  treatment,  E (1,  474)  = 

53.28,  MSE  = 1.616,  p < .000.  On  the  pretest  the  control  group  (x  = 1.28)  had  higher  scores  than  did  the 
experimental  group  (x  = 1.13).  However,  the  experimental  group  (x  = 2.73)  did  achieve  significantly  higher  scores 
as  compared  to  the  control  group  (x  = 1.54)  on  the  posttest. 

Using  the  scores  of  the  experimental  group,  females  and  males  were  analyzed  separately  to  obtain  more 
precise  data  on  the  gender  difference  of  the  improvement  from  instruction.  The  results  of  ANOVA  indicated  that 
there  were  significant  differences  for  gender  E (1,  231)  = 4.40,  p = .037,  and  test  occasion  E (1,  231)  = 79.6,  p < 

.000.  Eor  test  occasion,  subjects’  scores  were  significantly  higher  for  the  posttest  (x  = 2.35  for  males  and  x = 3.12 
for  females)  than  those  of  the  pretest  (x  = 1.16  for  males  and  x = 1.13  for  females).  The  scores  of  the  posttest 
revealed  that  the  females  statistically  outperformed  the  males.  Additionally,  ANOVA  yielded  a significant  two-way 
interaction  for  gender  factor  by  test  occasion,  E (1,  231)  = 5.05,  p = .03.  This  interaction  reflects  the  fact  that  the 
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female  group  received  more  benefits  for  understanding  basic  concepts  from  the  multimedia  program  than  the  male 
group.  What  these  two  results  indicate  is  that  students  working  with  the  multimedia  program  improved  their 
conceptual  understanding  regardless  of  the  gender  difference  and  that  females  received  slightly  higher  benefits. 

Table  1.  Descriptive  Statistics  of  Conceptual  Understanding  and  Problem-Solving  Tests  by  Gender 

Conceptual  Understanding  Problem  Solving 


n 

M 

SD 

n 

M 

SD 

Control 

Pretest 

119 

1.28  (26%) 

1.02 

119 

.55  (3%) 

.82 

Posttest 

119 

1.54  (31%) 

1.24 

119 

.80  (4%) 

.14 

Treatment 

Pretest 

119 

1.13(23%) 

.98 

119 

.42  (2%) 

.82 

Posttest 

118 

2.73  (54%)* 

1.72 

118 

2.92(15%)* 

2.84 

Treatment 

Pretest 

Male 

57 

1.16(23%) 

.88 

57 

.47  (2  %) 

.97 

Female 

61 

1.13(23%) 

1.06 

61 

.37  (1  %) 

.66 

Posttest 

Male 

56 

2.35  (47  %) 

1.39 

56 

2.57  (13%) 

2.57 

Female 

61 

3.12(62%)* 

1.89 

61 

3.28(16%) 

3.05 

Note.  * P<.05.  ns  vary  because  of  missing  data.  M:  mean.  SD:  Standard  Deviation.  Mean  proportion  of  correct 
responses  is  shown  in  parentheses. 


Problem  Solving 

With  regard  to  conceptual  understanding,  the  2 (treatment:  experimental  vs.  control)  x 2 (test  occasion: 
pretest  vs.  posttest)  ANOVA  yielded  significant  interaction,  F (1,  474)  = 149.44,  MSE  = 2.82,  p < .000,  in  problem 
solving  as  well.  While  the  experimental  group  (x  = .42)  performed  lower  than  did  the  control  group  (x  = .55)  in  the 
pretest,  the  experimental  group  (x  = 2.92)  had  significantly  higher  scores  than  did  the  control  group  (x  = .80)  in  the 
posttest.  In  the  analysis  of  the  gender  differences,  females  scored  higher  than  males  on  well-structured  problem 
solving  (x  = 2.57  for  males,  and  x = 3.28  for  females).  However,  ANOVA  indicates  this  gender  difference  is  not 
statistically  significant.  Both  the  female  and  male  students  in  the  experimental  group  significantly  outperformed 
those  of  the  control  group  on  well-structured  problem  solving  in  the  posttest.  In  both  groups  the  scores  of  the 
posttest  were  significantly  different  from  those  of  the  pretest,  F (1,  231)  = 84.6,  p < .000,  indicating  that  participants 
in  the  multimedia  program  improved  their  problem-solving  skills  after  instruction. 

Discussion  and  Implications 

As  noted  by  comparing  overall  results,  students  working  with  the  multimedia  program  improved  their 
conceptual  understanding  and  problem-solving  skills  after  instruction  regardless  of  their  gender.  The  findings  show 
that  the  multimedia  learning  environment.  Astronomy  Village,  designed  around  inquiry  is  an  effective  tool  for 
helping  students  learn  astronomy  content  and  problem-solving  skills.  It  was  speculated  that  having  students  engage 
in  inquiry -based  activities  similar  to  what  real-life  scientists  undertake  would  help  students  better  understand 
underlying  concepts  and  improve  their  problem  solving.  The  results  demonstrate  this  and  support  previous  research 
about  the  positive  effect  of  inquiry -based  approaches  incorporating  multimedia  technology  (Bybee,  1993;  Collins, 
1998;  Kuhn  et  ak,  1988;  Kuhn.D.  et  al.,  1988;  Mayer  et  ak,  1999;  Moore,  1993;  National  Research  Council,  1996; 
Pea,  1993;  Uno,  1990;  Windschitl,  2000). 

Moreover,  the  results  showed  that  female  students  outperformed  male  students  on  the  conceptual 
understanding  posttest.  This  finding  speculates  that  the  female  group  received  more  benefits  from  the  inquiry -based 
multimedia  program  than  the  male  group.  It  also  provides  empirical  support  that  female  students  received  more 
benefits  from  the  inquiry  activities  than  male  students  (Anderson,  1993;  Greenberg -Lake,  1991;  Hill  et  ak,  1990; 
Lock,  1992;  McLaren  & Gaskell,  1995;  Thomas,  1986).  Overall,  this  study  supports  inquiry -based  multimedia 
technology  as  a learning  and  teaching  tool  with  a high  potential  for  facilitating  students’  learning,  especially  for 
females. 

The  results  point  to  what  science  educators  can  do  to  improve  students’  problem-solving  skills,  particularly 
for  females  in  science.  The  findings  presented  in  this  study  also  indicate  that  the  inquiry -based  approaches  offer 
promising  methods  for  improving  problem-solving  skills  among  all  science  students,  especially  female  students.  It 
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will  be  a prescriptive  method  to  counter  the  obstacles  in  science  education  that  impinge  on  gender  equity  in  science 
instruction.  Future  empirical  studies  should  examine  whether  results  are  consistent  with  those  of  this  study. 


References 

American  Association  of  University  Women  [AAUW].  (1992).  how  schools  shortchange  girls:  A study  of 
major  findings  on  girls  and  education:  Washington  DC:  Author. 

Anderson,  B.  T.  (1993).  Minority  females  in  the  science  pipeline:  Activities  to  enhance  readiness, 
recruitment,  and  retention. /nitiaf/ves',  55(3),  31-37. 

Baxter,  G.  P.,  Glaser,  R.,  & Raghavan,  K.  (1993).  Analysis  of  cognitive  demand  in  selected  alternative 
science  assessments  (Vol.  ED  368  776).  Pittsburgh,  PA:  CRESST/Leaming  Research  and  Development  Center. 

Bredderman,  T.  (1982).  Activity  science-the  evidence  shows  it  matters.  Science  and  Children,  20,  39-41. 

Bredderman,  T.  (1983).  Effects  of  activity-based  elementary  science  on  student  outcomes:  A quantitative 
synthesis.  Review  of  Educational  Research,  55(4),  499-518. 

Bybee,  R.  W.  (1993).  Reforming  science  education:  Social  perspectives  and  personal  reflections.  New 
York:  Teacher  College  Press. 

Chiappetta,  E.,  & Russell,  J.  M.  (1982).  The  relationship  among  logical  thinking,  problem-solving 
instruction,  and  knowledge  and  application  of  Earth  science  subject  matter.  Science  Education,  66(1),  85-93. 

Clark,  H.  T.  (1990).  Recall  of  information  separating  two  inconsistent  propositions:  An  experimental  test 
of  the  cognitive  dumping  hypothesis.  Paper  presented  at  the  Eastern  Educational  Research  Association,  Clearwater, 
EL. 

Collins,  A.  (1998).  National  science  education  standards:  A political  document.  Journal  of  Research  in 
Science  Teaching,  34(1),  111-121 . 

Craig,  D.  V.  (1999,  June  22-24).  A league  of  their  own:  Gender,  technology,  and  instructional  practices. 
Paper  presented  at  the  National  Educational  Computing  Conference  proceedings,  Atlantic  City,  NJ. 

Doran,  R.  L.,  Boorman,  J.,  Chan,  E.,  & Hejaily,  N.  (1993).  Alternative  assessment  of  high-school 
laboratory  skills.  Journal  of  Research  in  Science  Teaching,  50(9),  1121-1131. 

Eccles,  J.  (1989).  Bringing  young  women  to  math  and  science.  In  M.  Crawford  & M.  Gentry  (Eds.), 

Gender  and  thought:  Psychological  perspectives  {pp.  36-58).  New  York:  Springer-Verlag. 

Ereedman,  M.  P.  (2001).  The  influence  of  laboratory  instruction  on  science  achievement  and  attitude 
toward  science  among  ninth  grade  students  across  gender  differences  [ED  454070].  Retrieved,  from  the  World 
Wide  Web: 

Gagne,  R.  M.  (1970).  The  conditions  of  learning  (2nd  Ed.).  New  York:  Holt,  Rinehart,  & Winston. 

Gall,  M.  D.,  Borg,  W.  R.,  & Gall,  J.  P.  (1996).  Educational  research:  An  introduction  (6th  ed.).  New  York: 
Longman  Publisher. 

Geban,  O.,  Askar,  P.,  & Ozkan,  I.  (1992).  Effects  of  computer  simulations  and  problem-solving  approaches 
on  high  school  students.  Journal  of  Educational  Research,  <S6(1),  5-10. 

Glasson,  G.  E.  (1989).  The  effects  of  hands-on  and  teacher  demonstration  laboratory  methods  on  science 
achievement  in  relation  to  reasoning  ability  and  prior  knowledge.  Journal  of  Research  in  Science  Teaching,  26(2), 
121-131. 

Greenberg -Lake,  T.  A.  G.  (1991).  Shortchanging  girls,  shortchanging  America.  Washington,  D.C.: 
American  Association  of  University  Women. 

Haury,  D.  L.  (1993).  Teaching  science  through  inquiry.  Columbus,  Ohio:  ERIC  Clearinghouse  for  science, 
mathematics,  and  environmental  education. 

Hill,  O.  W.,  Pettus,  W.  C.,  & Hedin,  B.  A.  (1990).  Three  studies  of  factors  affecting  the  attitude  of  blacks 
and  females  toward  the  pursuit  of  science  and  science-related  careers.  Journal  of  Research  in  Science  Teaching, 
27(4),  289-314. 

Hong,  N.  S.  (1998).  The  relationship  between  well-structured  and  ill -structured  problem  solving  in 
multimedia  simulation.  The  Pennsylvania  State  University,  State  College. 

Howard,  B.  C.,  McGee,  S.,  Shin,  N.,  & Regina,  S.  (2001).  The  triarchic  theory  of  intelligence  and 
computer-based  inquiry  learning.  Educational  Technology  Research  and  Development,  49(4),  49-69. 

Jenkins,  L.  B.,  & MacMonald,  W.  B.  (1989).  Science  teaching  in  the  spirit  of  science.  Issues  in  Science  and 
Technology,  5(3),  60-65. 

Jewett,  T.  O.  (1996).  "And  they  is  us":  Gender  issues  in  the  instruction  of  science.  Edwardsville,  II: 
Southern  Illinois  University,  Department  of  Curriculum  and  Instruction. 


401 


Jonassen,  D.  H.,  Beissner,  K.,  & Yacci,  M.  (1993).  Structural  Knowledge.  Hillsdale,  N.J.:  Lawrence 

Erlbaum. 

Kahle,  J.  B.  (1996).  Opportunities  and  obstacles:  Science  education  in  the  schools.  In  C.  Davis  & A.  B. 
Genorio  & C.  S.  Hollinshead  & B.  B.  Lazarus  & R.  P.M.  (Bds.),  The  equity  equation:  Fostering  the  advancement  of 
women  in  the  sciences,  mathematics,  and  engineering  (pp.  57-97).  San  Francisco:  Josey-Bass. 

Kirkpatrick,  H.,  & Cuban,  L.  (1998).  Should  we  be  worried?  What  the  research  says  about  gender 
differences  in  access,  use,  attitudes,  and  achievement  with  computers.  Educational  Technology,  iS(4),  56-61. 

Krajcik,  J.,  Blumenfeld,  P.  C.,  Marx,  R.  W.,  Bass,  K.  M.,  Fredricks,  J.,  & Soloway,  F.  (1998).  Inquiry  in 
project-based  science  classrooms;  Initial  attempts  made  by  middle  school  students.  The  Journal  of  the  Learning 
Science,  7(3/4),  313-350. 

Kuhn,  D.,  Amsel,  E.,  & O'Loughlin,  M.  (1988).  The  development  of  scientific  thinking.  San  Diego: 
Academic  Press. 

Kuhn.D.,  Amsel,  E.,  & O'Loughlin,  M.  (1988).  The  development  of  scientific  thinking.  San  Diego: 

Academic  Press. 

Lindberg,  D.  H.  (1990).  What  goes  'round  comes'  round  doing  science.  Childhood  Education,  67{2),  79-81. 

Littleton,  K.,  Light,  P.,  Joiner,  R.,  Messer,  D.,  & Barnes,  P.  (1992).  Pairing  and  gender  effects  in  children's 
computer-based  learning.  European  Journal  of  Psychology  of  Education.,  7(4),  311-324. 

Lock,  R.  (1992).  Gender  and  practical  skill  performance  in  science.  Journal  of  Research  in  Science 
Teaching,  29(3),  227-241. 

Mael,  F.  A.  (1998).  Single-sex  and  coeducational  schooling:  Relationships  to  socioemotional  and  academic 
development.  Review  of  Educational  Research,  68(2),  101-129. 

Matyas,  M.  L.  (1985).  Factors  affecting  female  achievement  and  interest  in  science  and  in  scientific 
careers.  In  J.  B.  Kahle  (Ed.),  Women  in  science:  A report  from  the  field  (pp.  27-48).  London:  Falmer  Press. 

Mayer,  R.  E.,  Moreno,  R.,  Boire,  M.,  & Vagge,  S.  (1999).  Maximizing  constmctivist  learning  from 
multimedia  communications  by  minimizing  cognitive  load.  Journal  of  Educational  Psychology,  97(4),  638-643. 

McLaren,  A.,  & Gaskell,  J.  (1995).  Now  you  see  it,  now  you  don't:  Gender  s an  issue  in  school  science.  In 
J.  Gaskell  & J.  Willensky  (Eds.),  Gender  in/forms  curriculum:  From  enrichment  to  transformation  (pp.  136-156). 
New  York:  Teachers  College  Press. 

Moore,  J.  A.  (1993).  Science  as  a way  of  knowing.  Cambridge,  MA:  Harvard  University  Press. 

National  Research  Council.  (1996).  National  Science  Educational  Standards.  Washington  DC.:  National  Academy 
Press. 

Pea,  R.  D.  (1993).  Seeing  what  we  build  together:  Distributed  multimedia  learning  environments  for 
transformative  communications  (special  issue).  The  Journal  of  the  Learning  Sciences,  3(3),  285-299. 

Pompea,  S.  M.,  & Blurton,  C.  (1995  Jan/Feb).  A walk  through  the  Astronomy  Village.  Mercury,  32-33. 

Robitaille,  D.  F.,  Schmidt,  W.  H.,  Raizen,  S.,  MCKnight,  C.,  Britton,  E.,  & Niclo,  C.  (1993).  Curriculum 
frameworks  for  mathematics  and  science.  Vancouver,  Canada:  Pacific  Educational  Press. 

Rosser,  S.  V.  (1990).  Female  friendly  service.  New  York:  Pergamon  Press. 

Scanlon,  E.  (2000).  How  gender  influences  learners  working  collaboratively  with  science  simulations. 
Learning  and  Instruction,  10,  463-481. 

Schibeci,  R.  A.,  & Riley,  J.  P.  (1986).  Influence  of  students'  background  and  perceptions  on  science  attitude 
and  achievement.  Journal  of  Research  in  Science  Teaching,  23(3),  177-187. 

Shakeshaft,  c.  (1995).  Reforming  science  education  to  include  females.  Theory  into  Practice,  34(\),  74-79. 

Shin,  N.,  Jonassen,  D.  H.,  & McGee,  S.  (in  press).  Predictors  of  well-structured  and  ill-structured  problem 
solving  in  an  astronomy  simulation.  Journal  of  Research  in  Science  Teaching. 

Shymansky,  J.  A.,  Hedges,  L.  V.,  & Woodworth,  G.  (1990).  A reassessment  of  the  effects  of  inquiry -based 
science  curricular  of  the  60's  on  student  performance.  Journal  of  Research  in  Science  Teaching,  27(2),  127-144. 

Simpson,  R.  D.,  & Oliver,  J.  S.  (1985).  Attitude  toward  science  and  achievement  motivation  profiles  of 
male  and  female  science  students  in  grades  six  through  ten.  Science  Education,  69(511-526). 

Sjoberg,  S.  (1993).  Gender  equality  in  science  classrooms.  In  B.  J.  Fraser  (Ed.),  Research  implications  for 
science  and  mathematics  teachers  (Vol.  I.  Key  center  monograph,  pp.  31-36). 

Third  International  Mathematics  and  Science  Study.  (1998).  Highlights  of  results  from  TIMSS:  Middle 
school  years  [On-line]. 

Thomas,  G.  E.  (1986).  Cultivating  the  interest  of  women  and  minorities  in  high  school  mathematics  and 
science.  Science  Education,  70(1),  31-43. 

Uno,  G.  (1990).  Inquiry  in  the  classroom.  Bioscience,  40(11),  841-843. 


402 


Valentine,  E.  F.  (1998).  Gender  differences  in  learning  and  achievement  in  mathematics,  science,  and 
technology  and  strategies  for  equity:  A literature  review.  Retrieved,  from  the  World  Wide  Web: 

Whitelock,  D.,  O'Shea,  T.,  Taylor,  J.,  Scanlon,  E.,  Clark,  P.,  & O'Malley,  C.  (1995).  Computer  support  for 
pupils  collaborating:  a classroom  case  study  on  collisions.  Paper  presented  at  the  Comupter  Support  for 
Collaborative  Learning,  Indiana. 

Windschitl,  M.  (2000).  Supporting  the  development  of  science  inquiry  skills  with  special  classes  of 
software.  Educational  Technology  Research  and  Development,  48(2),  81-95. 

Yager,  R.  E.  (1995).  Science/Technology/Society  and  learning.  Bulletin  of  Science,  Technology,  and 
Society,  75(5-6),  215-121 . 

Zietsman,  A.  L,  & Hewson,  P.  W.  (1986).  Effect  of  instruction  using  microcomputer  simulations  and 
conceptual  change  strategies  on  science  learning.  Journal  of  Research  in  Science  Teaching,  25(11),  27-39. 


403 


Technology  Competencies  of  Rural  School  District  Administrators  and 
Instructional  Technology  Decision  Making 


Marilyn  J.  Staffo 
Stillman  College 


Abstract 

This  research  study  examines  the  relationship  between  the  technology  competencies  of  rural  school 
administrators  and  instructional  technology  decision  making.  Superintendents,  assistant  superintendents,  and 
technology  coordinators  from  rural  Alabama  school  districts  were  surveyed  regarding  their  perceptions  of  the  types 
of  instructional  technology  decisions  made,  the  influence  of  government  programs  on  instructional  technology 
decisions,  and  the  instructional  technology  decision-making  process. 

Introduction 

This  study  examines  relationships  between  the  technology  competency  levels  of  rural  school  district 
administrators  and  the  administrators’  perceptions  regarding  instructional  technology  decisions  made  in  their 
district.  The  study  is  particularly  concerned  with  the  rural  administrators’  perceptions  related  to  the  influence  of 
government  policies  and  programs  on  instructional  technology  decisions  and  perceptions  related  to  the  personnel 
involved  in  making  instructional  technology  decisions. 

From  the  mid-1990s  through  2000  federal  government  policies  encouraged  and  supported  the  use  of 
instructional  technology  in  public  schools  in  the  United  States  (Blaschke,  1998).  Many  federal  instructional 
technology  programs  focused  on  alleviating  the  problems  resulting  from  the  digital  divide,  including  recognition  of 
the  digital  divide’s  impact  on  rural  schools  (Clinton,  2000;  Rosenthal,  2000).  Part  of  the  Clinton  administration's 
policy  regarding  the  digital  divide  was  to  encourage  partnerships  between  schools  and  businesses  or  other  members 
of  the  community.  This  study  examines  Alabama  rural  district  administrator’ s perceptions  of  the  influence  of  these 
federal  instructional  technology  policies  and  programs  begun  during  the  Clinton  administration  on  decisions  made  in 
the  decisions  made  in  their  districts. 

Characteristics  of  rural  schools  influence  district  level  instructional  technology  decisions.  Providing 
Internet  access  and  networks  for  instructional  use  has  been  a challenge  for  rural  school  districts.  These  districts  must 
contend  with  older  rural  facilities  that  often  have  problems  with  their  electrical  systems  and  climate  control,  along 
with  a lack  of  telephone  or  cable  access  to  classrooms  (Phelps,  Peach,  & Reddick,  1998).  Rural  school 
administrators  have  difficulty  finding  knowledgeable  technology  coordinators,  or  community  members,  who  can 
install  and  maintain  comp  heated  networks  or  train  teachers  to  use  the  network.  The  small  size  and  location  of  rural 
schools  often  hamper  their  ability  to  find  corporate  partners  to  assist  with  networking  projects.  Many  rural  school 
administrators  and  teachers  are  at  a disadvantage  in  obtaining  technology  grants  because  of  lack  of  time  due  to 
multiple  responsibilities  and  limited  grant  writing  experience.  A large  number  of  administrators  in  rural  schools,  like 
those  in  urban  and  suburban  schools,  received  their  undergraduate  and  graduate  education  before  computer 
technology  courses  were  a standard  part  of  the  college  curriculum.  This  becomes  a greater  problem  for  rural  school 
administrators;  because  in  small  schools,  administrators  have  only  a limited  number  of  fellow  administrators  or  staff 
members  to  call  on  to  keep  them  up-to-date  on  instructional  technology  (Jensen,  1998). 

William  Crumley  (1993)  has  conducted  one  of  the  few  studies  that  describes  the  instructional  technology 
decision-making  process  in  a rural  setting.  He  describes  decision  making  that  occurred  during  the  development  of  a 
comprehensive  technology  plan.  Crumley  was  selected  to  chair  a committee  with  the  goal  to  develop  a long-range 
comprehensive  technology  plan  to  serve  as  a guide  for  budget  and  curriculum  development.  He  chose  the  following 
committee  members:  an  elementary  teacher,  a middle  school  teacher,  two  high  school  teachers,  one  teacher  from 
adult  education,  one  school  board  member,  one  administrator,  and  two  representatives  from  the  community.  The 
study  demonstrates  how  instructional  technology  decisions  can  be  made  in  a shared  decision-making  manner 
showing  the  involvement  of  a variety  of  stakeholders  in  the  group  (Crumley,  1993). 
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Methodology 

This  study  examines  the  following  research  question:  Is  there  a relationship  between  the  technology 
competencies  of  the  rural  district  administrators  and  perceptions  of  decisions  made  concerning  instructional 
technology?  Survey  research  was  conducted  to  examine  this  question.  The  population  surveyed  consisted  of 
Alabama  rural  school  district  superintendents,  assistant  superintendents,  and  technology  coordinators.  Rural  school 
districts  were  selected  because  they  are  one  of  the  groups  specifically  targeted  by  recent  government  instructional 
technology  programs.  The  districts  chosen  for  the  survey  were  identified  as  rural  based  on  National  Center  for 
Education  Statistics  (NCES)  information  (National  Center  for  Education  Statistics,  2000).  NCES  identifies  33 
Alabama  school  districts  as  rural.  These  districts  are  located  in  three  types  of  communities:  high  growth 
communities,  moderate  growth  communities,  and  high  child  poverty  rates  and/or  lack  of  economic  growth 
communities.  The  rural  districts  have  98  administrators  in  the  positions  of  superintendent,  assistant  superintendent, 
or  technology  coordinator.  Surveys  were  sent  to  the  entire  population  since  it  is  small.  Of  the  98  surveys  mailed  to 
administrators  in  Alabama  rural  school  districts,  65  responded  for  a return  rate  of  66%.  Demographic  data  is 
displayed  in  Table  1. 


Table  1 Frequency  and  Percentage  of  Respondent  Demographic  Data 


Demographic  data 

Freauencv 

Percent 

Positions 

Superintendents 

22 

34% 

Assistant  Superintendents 

19 

29% 

Technology  Coordinators 
Years  as  administrator  or 

24 

37% 

technolosv  coordinator  in  district 

New-3 

23 

35% 

4-10 

27 

42% 

More  than  10 

15 

23% 

Internet  and  e-mail  competency 

None 

1 

2% 

Low 

6 

9% 

Moderate 

27 

42% 

High 

Use  of  technology  for  instruction  competency 

31 

48% 

None 

1 

2% 

Low 

11 

17% 

Moderate 

31 

48% 

High 

Knowledge  of  distance  education  competency 

22 

34% 

None 

3 

5% 

Low 

25 

39% 

Moderate 

24 

37% 

High 

13 

20% 

Knowledge  of  networking  equipment  and  services 
competency 


None 

5 

8% 

Low 

20 

31% 

Moderate 

23 

36% 

High 

17 

26% 

Setting 

High  growth 

18 

28% 

Moderate  growth 

26 

40% 

High  child  poverty  rates  and/or  lack  of  economic  growth 

21 

32% 

As  Table  1 shows  the  65  respondents  to  the  survey  held  positions  as  superintendents,  assistant 
superintendents  or  technology  coordinators  in  rural  Alabama  school  districts.  All  three  positions  were  well 
represented  in  the  responses.  The  greatest  number  of  responses,  24,  came  from  technology  coordinators.  Slightly 
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fewer  responses,  22,  came  from  superintendents.  The  least  number  of  responses,  19,  came  from  assistant 
superintendents. Table  1 also  shows  that  most  of  the  participants  had  worked  less  than  ten  years  as  an  administrator 
in  their  district.  The  greatest  number  (42%)  had  between  four  and  ten  years  of  experience  as  an  administrator  in  the 
district.  Slightly  more  than  one-third  (35%)  had  less  than  one  year  and  up  to  three  years  as  an  administrator  in  the 
district.  Only  15  of  the  participants  had  spent  more  than  ten  years  in  a district  level  administrative  position  in  the 
school  district  where  they  were  employed. 

Demographic  data  were  collected  concerning  the  technology  competency  of  rural  district  level 
administrators.  Respondents  were  asked  to  provide  a self-assessment  of  their  level  of  technology  competency  in  four 
areas.  They  chose  from  "none,"  "low,"  "moderate,”  and  "high”  to  indicate  their  competency  level  in  the  use  of 
Internet  and  e-mail,  the  use  of  technology  for  instruction,  their  knowledge  of  distance  education,  and  their 
knowledge  of  networking  equipment  and  services.  Nearly  half  (48%)  of  the  respondents  indicated  a high  level  of 
competency  in  the  use  of  the  Internet  and  e-mail.  An  equal  number  indicated  a moderate  level  of  competency  in  the 
use  of  technology  for  instruction.  Slightly  less  (42%)  indicated  that  they  had  a moderate  level  of  competency  in  the 
use  of  Internet  and  e-mail.  Approximately  one-third  of  the  administrators  indicated  a high  level  of  competency  in  the 
use  of  technology  for  instruction.  Slightly  more  than  one-third  indicated  that  they  had  a moderate  knowledge  of 
distance  education  technology  and  a moderate  level  of  knowledge  of  networking  equipment  and  services.  A low 
competency  level  regarding  knowledge  of  distance  education  technology  was  chosen  by  39%  of  the  respondents  and 
a low  competency  level  regarding  knowledge  of  networking  equipment  and  services  was  chosen  by  31%  of  the 
respondents.  Less  than  10%  of  the  administrators  indicated  that  they  had  no  competency  concerning  knowledge  of 
networking  equipment  and  services.  The  technology  competency  levels  of  administrators  from  Alabama  rural  school 
districts  are  found  in  Table  1.  Data  concerning  technology  competency  levels  were  collapsed  for  purposes  of 
analysis.  Few  respondents  indicated  “none”  for  their  competency  level  in  the  four  competency  areas.  For  this  reason 
the  "none"  and  "low"  competency  levels  were  combined. 

Spearman  rank-order  correlation  analysis  was  conducted  on  the  data  collected.  This  procedure  was  used  to 
determine  whether  there  are  relationships  among  position,  technology  competencies,  and  setting.  Spearman  rank- 
order  correlation  analysis  revealed  a significant  relationship  between  technology  competency  levels  and  positions 
held.  Significant  relationships  were  found  for  position  and  competencies  in  (a)  the  use  of  the  Internet  and  e-mail  (js 
(63)  =.313,  2.<.05),  (b)  the  use  of  technology  for  instruction  (Es(63)  =.263,  g.<.05),  and  (c)  knowledge  of  distance 
education  (is(63)  =.251,  p<.05),  and  (d)  knowledge  of  networking  (is(63)  =.511,  p<.05). 


Table  2 Frequency  and  Percentage  of  Competency  Level  Responses  for  Uses  and  Knowledge  of  Technology  by 
Positions  Field 


Competency 

Superintendents 
Number  Percent 

Assistant 
Superintendents 
Number  Percent 

Technology 
Coordinators 
Number  Percent 

Use  of  Internet  and  e-mail* 

None/low 

3 

14% 

4 

21% 

0 

0% 

Moderate 

11 

50% 

9 

47% 

7 

29% 

High 

8 

36% 

6 

32% 

17 

71% 

Use  of  technology  for  instruction* 

None/low 

4 

18% 

6 

32% 

2 

8% 

Moderate 

13 

59% 

9 

47% 

9 

38% 

High 

5 

23% 

4 

21% 

13 

54% 

Knowledge  of  distance  education** 
None/low 

Moderate 

12 

55% 

9 

47% 

7 

29% 

High 

7 

31% 

8 

42% 

9 

38% 

3 

14% 

2 

11% 

8 

33% 

Knowledge  of  networking  equipment 
and  services** 

None/low 

11 

50% 

13 

68% 

1 

4% 

Moderate 

10 

45% 

3 

16% 

10 

42% 

High 

1 

5% 

3 

16% 

13 

54% 

Note.  * p<.05,  **p<.01 


406 


An  examination  of  the  data  in  Table  2 indicates  that  approximately  twice  as  many  technology  coordinators 
indicate  a high  level  of  competency  in  the  use  of  Internet  and  e-mail  and  in  the  use  of  technology  for  instruction 
compared  to  superintendents  or  assistant  superintendents  for  this  level  of  competency.  A higher  percentage  of 
assistant  superintendents  than  either  superintendents  or  technology  coordinators  chose  “none”  or  “low”  levels  of 
competency  in  all  of  the  competency  areas  except  knowledge  of  distance  education.  This  was  especially  true  in  the 
knowledge  of  networking  equipment  and  services.  Few  technology  coordinators  chose  “none”  or  “low”  for  their 
level  of  competency  in  any  of  the  competency  areas.  Knowledge  of  distance  education  was  the  only  competency 
area  in  which  more  than  25%  of  the  technology  coordinators  indicate  a “low”  level  of  competency.  This  was  still 
better  than  the  competency  levels  indicated  by  other  administrators.  The  percent  of  technology  coordinators  who 
indicate  a high  level  of  competency  in  knowledge  of  networking  equipment  and  services  is  more  than  ten  times 
higher  than  for  superintendents  and  nearly  four  times  higher  than  for  assistant  superintendents. 

Spearman  rank-order  correlation  analysis  revealed  a relationship  between  technology  competency  levels  of 
administrators  and  the  setting  of  the  rural  school  district.  Spearman  rank-order  correlation  analysis  revealed  a 
significant  relationship  between  the  rural  school  district  setting  and  competency  in  the  use  of  e-mail  and  the  Internet 
(Es(63)  =.394,  g.<.05),  use  of  instructional  technology  (rs(63)  =.265,  p<.05),  knowledge  of  distance  education  (rs(63) 
=.397,  p<.05),  and  knowledge  of  networking  (rs(63)  =.360,  p<.05). 

Table  3 Frequency  and  Percentage  of  Competency  Levels  by  Setting  of  the  Rural  School  District 

Competency  High  growth  Moderate  growth  Child  poverty  and/or 

lack  of  growth 

Number  Percent  Number  Percent  Number  Percent 


Use  of  E-mail  and  the  Internet** 


None/low 

3 

16% 

4 

15% 

0 

0% 

Moderate 

10 

56% 

12 

46% 

5 

24% 

High 

5 

28% 

10 

39% 

16 

76% 

Use  of  Instructional  Technology* 

None/low 

5 

28% 

7 

30% 

0 

0% 

Moderate 

7 

39% 

14 

52% 

10 

48% 

High 

6 

33% 

5 

18% 

11 

52% 

Knowledge  of  Distance  Education** 

None/low 

10 

56% 

16 

62% 

2 

10% 

Moderate 

5 

28% 

9 

35% 

10 

48% 

High 

3 

16% 

1 

3% 

9 

42% 

Knowledge  of  networking** 

None/low 

12 

67% 

8 

31% 

5 

24% 

Moderate 

4 

22% 

12 

46% 

7 

33% 

High 

2 

11% 

6 

23% 

9 

43% 

Note.  * p<.05,  **  p<.01 

Table  3 shows  that  administrators  from  school  districts  located  in  areas  with  high  child  poverty  rates  and/or  lack  of 
economic  growth  were  more  likely  to  indicate  a high  level  of  competency  in  the  use  of  e-mail  and  the  Internet,  the 
use  of  instructional  technology,  and  knowledge  of  networking  than  leaders  from  moderate  growth  or  high  growth 
settings.  Table  3 also  shows  that  administrators  from  districts  in  high  growth  areas  are  more  likely  to  indicate  a 
moderate  level  of  competency  in  the  use  of  e-mail  and  the  Internet  than  those  from  the  other  types  of  settings. 
Leaders  from  high  growth  and  moderate  growth  settings  are  more  likely  to  indicate  “none”  or  “low”  competency  in 
knowledge  of  distance  education  than  those  from  high  child  poverty  or  lack  of  growth  settings.  Participants  from 
high  growth  settings  are  more  likely  to  indicate  “none”  or  “low”  competencies  in  knowledge  of  networking  than 
those  from  either  moderate  growth  settings  or  those  from  settings  with  high  child  poverty  or  lack  of  growth  settings. 
The  data  analysis  shows  that  there  is  a relationship  between  administrator’s  technology  competency  levels  and  the 
demographic  factors  of  position  and  setting  of  the  district. 
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The  Spearman  rank -order  correlation  analysis  revealed  a significant  relationship  between  years  as  a district 
level  administrator  in  the  district  and  competency  in  the  knowledge  of  distance  education  (rs(63)  =.306,  p<.05).  The 
significant  results  are  shown  in  Table  4. 

Table  4 Frequency  and  Percentage  of  Competency  Level  Reponses  for  Knowledge  of  Distance  Education  by  Years 
as  an  Administrator  in  the  District 

Competency  New  to  3 years  4 to  10  years  More  than  10  years 

Number  Percent 

Number  Percent  Number  Percent 

Knowledge  of  Distance  Education* 


None/low 

14 

61% 

11 

41% 

3 

20% 

Moderate 

6 

26% 

11 

41% 

7 

47% 

High 

3 

13% 

5 

18% 

5 

33% 

Note.  * p<.05 

Administrators  who  had  three  or  fewer  years  of  district  level  experience  in  the  district  are  much  more  likely 
to  indicate  a “low”  level  of  knowledge  concerning  distance  education  than  those  with  more  years  of  leadership 
experience  in  the  district.  As  can  be  seen  in  the  table  above,  administrators  with  ten  or  more  years  of  experience  in 
the  district  are  likely  to  indicate  a “moderate”  or  “high”  level  (80%)  of  competency  for  knowledge  of  distance 
education.  The  data  analysis  shows  that  there  is  a relationship  between  administrator’s  technology  competency 
levels  and  demographic  factors  such  as  position,  setting  of  the  district,  and  years  as  an  administrator.  The  next 
section  of  the  study  examines  whether  the  relationships  regarding  technology  competency  also  extend  to  perceptions 
regarding  aspects  of  the  instructional  technology  decision-making  process. 

Next,  the  study  examines  whether  the  relationships  regarding  technology  competency  also  extend  to 
perceptions  regarding  aspects  of  the  instructional  technology  decision-making  process.  The  first  perceptions  to  be 
examined  are  those  concerning  the  influence  government  instructional  technology  programs  on  district  instructional 
technology  decisions  and  on  the  acquisition  of  technology  resources  and  services.  Six  items  on  the  survey  refer  to 
the  influence  of  government  programs  and  policies.  For  these  items  respondents  were  asked  to  indicate  their 
agreement  or  disagreement  concerning  the  influence  of  government  overall  and  on  the  acquisition  of  various  types 
of  resources  or  services  regarding  decisions  made  during  the  last  four  years  (1997-2001).  The  results  of  the  items 
related  to  this  question  were  totaled.  Higher  totals  indicated  stronger  agreement  concerning  government  influence 
on  district  instructional  technology  decisions.  Responses  to  the  totaled  items  are  described.  One-way  analysis  of 
variance  procedures  were  conducted.  The  significant  ANOVA  results  are  also  described,  including  results  of  follow- 
up tests  that  indicate  which  groups  showed  significance.  Descriptive  results  from  the  Items  Related  to  the  related  to 
this  area  of  the  study  appear  in  Table  5. 


Table  5 Descriptive  Data  for  the  Items  Related  to  the  Influence  of  Government  Programs  on  Decisions  Regarding 


12 

13 

15 

16 

Totals  for  items 
17  18  19  20 

21 

22 

23 

24 

Frequency 

1 

1 

2 

1 

7 8 5 2 

11 

7 

7 

13 

Percentage 

2% 

2% 

3% 

2% 

10%  12%  7%  3% 

17%  10% 

10% 

20% 

Note.  Highest  possible  total  for  six  items  in  group  =24. 


As  Table  5 shows  the  totals  for  the  items  related  to  the  influence  of  government  programs  regarding  the 
acquisition  of  resources  and  services  range  from  12  to  24.  The  percentages  for  each  of  the  totals  range  from  2%  to 
20%.  The  percentages  related  to  totals  20  through  24  account  for  60%  of  the  responses  for  this  totaled  group  of 
items. 

One-way  ANOVAs  were  conducted  for  this  data.  Significant  differences  were  found  among  competency 
levels  for  knowledge  of  distance  education  (F(2,62)  =3.558,  p<.05)  after  one-way  ANOVAs  were  run.  Tukey's  HSD 
was  used  to  determine  the  nature  of  the  differences  between  levels.  Tukey’s  HSD  reveals  that  respondents  with  a 
high  level  of  distance  education  knowledge  competency  (M  = 22.00,  SD  = 1.58)  have  a significantly  higher  level  of 
agreement  concerning  the  influence  of  government  programs  on  decisions  regarding  the  acquisition  of  resources  and 
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services  than  respondents  with  none  to  low  levels  of  dis  tance  education  knowledge  competency  (M  = 19.46,  SD  = 
3.25). 

This  study  also  examined  how  rural  administrators  perceive  the  influence  of  government  programs  on  the 
development  of  instructional  technology  partnerships.  Two  items  on  the  survey  addressed  this  area  of  the  study.  For 
these  items  respondents  were  asked  to  indicate  their  agreement  or  disagreement  concerning  government 
encouragement  on  the  formation  of  technology  partnerships  community  groups  or  technology  vendors.  The  results 
of  the  two  items  related  to  this  question  were  totaled.  A higher  total  indicates  stronger  agreement  that  government 
programs  influenced  the  formation  of  partnerships.  One-way  ANOVAs  were  conducted.  Descriptive  data  for  these 
totaled  items  are  found  in  Table  6.  The  table  shows  that  totals  for  the  items  range  from  two  through  eight.  Eight  is 
the  highest  possible  response  for  this  compound  group.  Percentages  range  from  2%  to  35%. 

Table  6 Descriptive  Data  for  Items  Related  to  the  Influence  of  Government  Programs  on  the  Formation  of 
Partnerships 


Totals  for  items 

2 

4 

5 

6 

7 

8 

Frequency 

1 

14 

15 

23 

8 

4 

Percentage 

2% 

22% 

23% 

35% 

12% 

6% 

Note.  The  highest  possible  total  for  group=8 


One-way  ANOVAs  were  run  for  the  total  of  items  for  the  influence  of  government  programs  on  the 
formation  of  partnerships.  Significant  differences  were  found  among  competency  levels  for  knowledge  of  distance 
education  (F(2,62)  =3.558,  p<.05).  Tukey's  HSD  was  used  to  determine  the  groups  showing  differences.  This 
analysis  revealed  that  respondents  with  a high  level  of  distance  education  knowledge  competency  (M  = 6.38,  SD  = 
1.26)  have  a significantly  higher  level  of  agreement  concerning  the  influence  of  government  on  the  formation  of 
technology  partnerships  than  respondents  with  none  to  low  levels  of  distance  education  knowledge  competency  (M 
= 5.32,  SD  = 1.25)  as  well  as  for  those  with  a moderate  level  of  distance  education  knowledge  (M  = 5.29,  SD  = 

1.00). 

The  next  area  of  interest  in  this  study  is  whether  there  is  a relationship  between  rural  district  administrators’ 
technology  competency  level  and  perceptions  concerning  the  participants  in  the  instructional  technology  decision- 
making process.  Results  relating  to  this  part  of  the  study  can  be  found  in  four  items  on  the  survey.  Descriptive  data 
for  the  items  related  to  this  area  of  interest  are  displayed  in  Table  7. 


Table  7 Descriptive  Data  for  Participants  in  the  Decision-making  Process 


Item 

N 

Number 
AN  S 

F 

N 

Percent 
AN  S 

F 

Mean 

Standard 

Deviation 

Groups  or  teams 

1 

4 24 

35 

1.5 

6.2  36.9 

53.8 

3.4531 

.6884 

Teachers 

1 

2 30 

31 

1.5 

3.1  46.2 

47.7 

3.4219 

.6376 

Students 

6 

35  20 

3 

9.2 

53.8  30.8 

4.6 

2.3125 

.7099 

Community 

members 

1 

23  28 

12 

1.5 

35.4  43.1 

18.5 

2.7969 

.7597 

Note.  N=never  AN=almost  never  S=sometimes  F=frequently.  The  scale  was  1-4.  A higher  mean  indicates 
more  frequent  participation. 


Table  7 shows  that  53.8%  of  the  rural  administrators  perceived  that  groups  or  teams  frequently  made 
instructional  technology  decisions.  Groups  or  teams  were  perceived  to  make  these  decisions  sometimes  by  36.9%. 
Only  7.9  % indicated  that  groups  or  teams  were  never  or  almost  never  involved  in  instructional  technology 
decisions.  (M=3.4531,  SD  = .6884). 

As  the  table  above  shows,  according  to  46.2%  of  the  participants,  teachers  were  sometimes  involved  in  the 
instructional  technology  decision-making  process.  Nearly  the  same  number  of  participants  (47.7%)  indicates  that 
teachers  were  frequently  involved  in  the  decision-making  process.  Less  than  one-third  (30.8%)  indicates  that 
students  were  sometimes  involved  in  the  instructional  technology  decision-making  process.  Slightly  more  than  twice 
as  many  (63%)  indicate  that  students  were  never  or  almost  never  involved  in  this  process.  The  respondents  believe 
that  community  members  were  mo  re  frequently  involved  in  the  instructional  technology  decision-making  process 
than  students.  Data  analysis  shows  that  18.5%  believe  that  community  members  were  frequently  involved  in  the 
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process,  43.1%  believe  that  community  members  were  sometimes  involved  in  the  process,  and  35.4%  believe  that 
community  members  were  never  or  almost  never  involved  in  the  process.  (The  findings  are:  teacher  involvement:  M 
=3.4219,  SD  = .6376;  student  involvement:  M = 2.3125,  SD  =.7099;  community  involvement:  M = 2.7969,  SD  = 
.7597.) 

Chi-square  tests  of  independence  were  conducted  for  competency  levels  of  rural  administrators  and  the 
items  concerning  who  was  involved  in  the  decision-making  process.  A significant  interaction  was  found  (9)  = 
21.970,  p.<.05)  for  competency  levels  regarding  knowledge  of  distance  education  and  how  frequently  teachers 
participated  in  the  instructional  technology  decision-making  process.  Participants  with  moderate  or  high  levels  of 
competency  regarding  knowledge  of  distance  education  are  more  likely  to  indicate  that  teachers  were  frequently 
involved  in  the  decision-making  process  than  those  with  low  levels  of  competency  in  the  knowledge  of  distance 
education. 

Chi-Square  and  ANOVA  analyses  conducted  for  this  study  show  that  one  group  of  respondents, 
administrators  with  a high  competency  level  concerning  knowledge  of  distance  education,  have  significantly 
different  perceptions  about  the  instructional  technology  decision-making  process  compared  to  other  administrators. 
The  respondents  in  the  group  with  a high  level  of  distance  education  competency  reveal  a stronger  agreement  than 
other  administrators  with  items  related  to  the  influence  of  government  programs  on  instructional  technology 
decisions  and  on  the  formation  of  partnerships.  One-way  ANOVAs  found  significant  differences  for  items  related  to 
items  asking  about  the  influence  of  government  programs  on  the  formation  of  partnerships.  Table  8 displays 
descriptive  data  concerning  this  group  of  administrators. 

Table  8 Descriptive  Data  for  Administrators  with  a Hif^h  Competency  Level  for  Knowledge  of  Distance  Education 

Demographic  Area Number Percent 

Positions 

Superintendent 
Assistant 

Superintendent 
Technology 

Coordinator 
Years  of  Experience 
New- 3 
4-10 

More  than  1 0 
Settings 

High  growth 
Moderate  growth 
High  child  poverty  rates 
and/or  lack  of  economic 
growth 
Note.  N=13 

The  group  of  administrators  with  a high  competency  level  in  the  area  of  distance  education  consists  of  13 
respondents  or  20%  of  the  survey  participants.  The  group  is  composed  of  eight  technology  coordinators,  three 
superintendents,  and  two  assistant  superintendents.  When  compared  with  the  entire  group  of  participants  this  group 
has  more  administrative  experience  in  their  district.  Ten  of  the  13  group  members  have  four  or  more  years  of 
experience  as  an  administrator  in  their  district,  and  five  participants  from  this  group  have  more  than  ten  years  of 
administrative  experience  in  their  district.  Interestingly,  69%  of  this  group  is  composed  of  administrators  from 
districts  in  areas  with  high  child  poverty  rates  and/or  lack  of  economic  growth. 

Following  are  the  significant  findings  from  the  data  analyses: 

1.  Administrators  from  settings  with  high  child  poverty  rates  and/or  lack  of  economic  growth  have 
significantly  higher  technology  competency  levels  than  those  from  moderate  or  high  growth 
settings. 

2.  Technology  coordinators  have  significantly  higher  technology  competency  levels  than 
superintendents  and  assistant  superintendents. 

3.  Administrators  with  more  years  of  administrative  experience  in  their  district  have  significantly 
higher  levels  of  competency  for  the  knowledge  of  distance  education. 


23% 

15% 


62% 


23% 

39% 

39% 


23% 

8% 

69% 
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4.  Administrators  with  a high  level  of  competency  for  knowledge  of  distance  education  are  more 
likely  than  administrators  with  lower  competency  levels  for  knowledge  of  distance  education  to 
agree  that  government  programs  influenced  instructional  technology  decisions  regarding  the 
acquisition  of  resources  and  services. 

5.  Administrators  with  a high  level  of  competency  for  knowledge  of  distance  education  are  more 
likely  than  administrators  with  lower  competency  levels  for  knowledge  of  distance  education  to 
agree  that  government  programs  encouraged  the  formation  of  technology  partnerships. 

6.  Administrators  with  moderate  and  high  competency  levels  for  knowledge  of  distance  education 
indicate  that  teachers  more  frequently  participate  in  the  instructional  technology  decision-making 
process  than  administrators  with  lower  levels  of  competency  for  knowledge  of  distance  education. 

7.  Perceptions  differed  for  administrators  with  different  levels  of  competency  levels  for  knowledge 
of  distance  learning  regarding  the  influence  of  government  programs  on  instructional  technology 
decisions  related  to  the  acquisition  of  resources  and  services  and  the  formation  of  technology 
partnerships. 

Conclusions  and  Recommendations 

Most  Alabama  rural  districts  have  district  level  administrators  who  are  competent  in  the  skills  and 
knowledge  of  technology  needed  to  make  instructional  technology  decisions.  This  is  particularly  true  of  technology 
coordinators  in  Alabama  mral  school  districts.  The  only  exception  to  high  levels  of  instructional  technology 
competency  is  the  knowledge  of  distance  education.  This  is  an  area  where  many  Alabama  rural  administrators  need 
additional  technology  staff  development.  This  study  found  that  among  the  administrators  who  participated  in  this 
study,  there  is  a small  core  group  of  district  level  administrators  (20%)  who  have  a high  level  of  knowledge  in  the 
area  of  distance  education. 

The  study  also  shows  that  the  rural  administrators  with  a high  level  of  competency  concerning  the 
knowledge  of  distance  education  perceive  a greater  influence  of  government  influence  in  some  areas  of  instructional 
technology  decision  making.  Since  many  distance  education  programs  are  government  sponsored  or  require 
government  funding,  the  leaders  who  have  high  levels  of  competence  in  this  area  have  greater  experience  over  an 
extended  time  period  with  government  instructional  technology  programs.  This  is  likely  to  be  why  they  perceive  a 
greater  influence  from  government  programs  on  instructional  technology  decisions  than  leaders  without  as  high  a 
competency  level  in  this  area.  In  addition,  the  study  shows  that  a high  percentage  of  the  leaders  with  a high  level  of 
competency  in  knowledge  of  distance  education  work  in  districts  with  high  child  poverty  rates  and/or  a lack  of 
economic  growth.  This  is  a positive  conclusion  because  this  group  of  leaders  is  capable  of  helping  their  districts 
located  in  poorer  settings  obtain  instructional  resources  for  their  students  through  distance  education  that  are  not 
available  otherwise  in  their  communities. 

Most  administrators  in  rural  Alabama  school  districts  are  competent  in  areas  of  technology  related  to 
instructional  technology  decision-making,  with  the  exception  of  knowledge  of  distance  education.  In  addition,  the 
leaders  with  a high  competency  level  concerning  the  knowledge  of  distance  education  perceive  a greater  influence  of 
government  programs  on  instructional  technology  decisions  possibly  related  to  greater  experience  over  an  extended 
time  period  with  government  sponsored  distance  education  activities.  Rural  districts  located  in  poorer  settings  benefit 
from  the  finding  that  administrators  from  settings  with  higher  child  poverty  rates  and/or  lack  of  economic  growth 
have  higher  competency  levels  regarding  the  use  and  knowledge  of  technology  than  leaders  from  other  settings. 

It  is  recommended  that  qualitative  studies  be  conducted  to  learn  about  why  there  are  differences  in  technology 
competency  levels  for  rural  administrators  in  poorer  settings.  Qualitative  researchers  should  conduct  interviews  and 
make  observations  that  delve  more  deeply  into  the  formal  and  informal  technology  training  and  experiences  of  a 
sample  of  district  level  administrators  from  the  various  settings  to  determine  why  these  differences  exist. 

Further  study  is  recommended  to  compare  the  technology  competency  levels  of  administrators  in  various 
settings  and  geographic  areas.  Future  study  is  needed  to  look  for  changes  in  findings  as  more  district  level 
administrators  participate  in  technology  professional  development  programs  that  provide  for  the  acquisition  of 
additional  technology  competencies  and  improvement  in  the  competency  levels  examined  in  this  study. 
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Abstract 

Using  the  lens  of  Mary  Douglas  ’ Grid  and  Group  Typology,  an  organizational  culture  model,  the  purpose 
of  this  case  study  was  to:  (1)  describe  the  organizational  context  of  two  colleges;  (2)  explain  the  influences  on 
individual  faculty  members'  preferences  toward  IT  use;  and  (3)  explain  the  interrelationship  of  culture  and  the 
decision  process  to  implement  IT  use  in  curricula.  Through  this  study,  the  College  of  Human  Ecology  was  best 
described  as  a Corporate  (high  grid/high  group)  culture,  while  the  College  of  Veterinary  Studies  best  fit  in  the 
Collectivist  (low  grid/high  group)  category.  The  findings  also  suggested  patterns  of  barriers  and  incentives  related 
to  IT  use  in  each  college,  reinforcing  the  reviewed  literature.  Also  indicated  was  that  the  cultural  bias  of  many  of  the 
faculty  members  in  CHE  is  individualistic  in  perceptions  of  incentives  to  use  IT.  This  individualistic  cultural  bias  is 
juxtaposed  against  the  overall  cultural  identity,  which  is  Corporate,  or  Hierarchical.  This  can  be  one  explanation 
for  apparent  discrepancies,  conflict,  and  dissatisfaction  among  the  cultural  members  of  the  college.  In  contrast,  the 
cultural  bias  of  the  faculty  and  the  cultural  identity  of  the  college  in  CVS  are  more  in  harmony.  For  example, 
incentives  are  also  intrinsic  and  self  defined,  but  this  low-grid  characteristic  is  in  sync  with  the  Collective 
environment  of  the  college.  Because  of  this  alignment,  there  are  less  discrepancies,  conflict,  and  dissatisfaction  in 
relation  to  IT  use.  The  findings  suggested  conflict  between  cultural  biases  of  individual  faculty  members  and  actual 
cultural  identity  of  the  college  as  a key  mitigating  factor  to  IT  implementation. 

Introduction 

While  the  availability  of,  familiarity  with,  and  pressures  to  use  instructional  technology  (IT)  have  greatly 
increased,  higher  education  faculty  members  are  using  IT  tools  at  levels  and  in  ways  that  can  hardly  be  considered 
revolutionary  (Baldwin,  1998;  McArthur  and  Lewis,  1998;  McMillan  and  Hyde,  2000;  Green  & Gilbert,  1995; 
Gilbert,  1995).  Understanding  why  some  faculty  members  embrace  IT,  while  others  in  the  same  organization  do  not, 
is  a crucial  aspect  of  implementation  and  the  focus  of  this  study. 

Noted  barriers  to  IT  implementation  include  techno-phobia,  fear  of  and  resistance  to  change,  pedagogical 
or  philosophical  incongruity,  deficient  monetary  incentives,  disempowerment,  and  frustration  with  new,  difficult, 
and  unreliable  technologies  (Rutherford  & Grana,  1995;  Derco,  1999;  Gilbert  & Green,  1994;  Bollentin,  1998; 
DeSieno,  1995;  Ganzert  & Watkins,  1997;  Kussmaul,  1996;  Lee,  1996;  McCollum,  1998;  Nantz  & Lundgren, 

1998).  Each  of  these  explanations  has  some  degree  of  merit,  but  an  all-encompassing,  holistic  perspective  to  the 
dilemma  can  be  found  in  Cultural  Theory.  This  study  posits  that  a university’s  organizational  culture  has  influence 
on  the  preferences  and  practices  of  IT  use  among  faculty  members.  This  notion  implies  that  IT  is  a sociocultural 
construct  rather  than  merely  a technical  construct,  and  cultural  context  exerts  tremendous  influence  over  individual 
choice  and  preferences  (Goodman,  Griffith,  & Fenner,  1990;  Sproull  & Goodman,  1990;  Susman,  1990;  Shields, 
1995). 

A common  tendency  is  to  limit  the  focus  of  IT  use  to  the  tangible  aspect  of  technology  tools  and  expect 
their  availability  and  apparent  benefits  to  lead  faculty  members  to  appropriate  application  and  practice.  This 
viewpoint  leaves  out  the  two  very  important  aspects  of  social  and  cultural  measures  (Pacey,  1983).  If  we  merely 
consider  the  functional  application  of  IT  without  considering  cultural  and  organizational  aspects,  we  will  continue  to 
experience  an  inability  to  fully  explain  the  dichotomy  between  faculty  members  who  use  IT  and  faculty  members 
who  do  not. 

The  purpose  of  this  study  is  to  explain,  in  terms  of  grid  and  group,  the  organizational  context  of  two 
colleges  within  a large,  four-year,  research  institution  and  the  interrelationship  of  context  and  faculty  IT  preferences 
and  practices. 

Conceptual  Framework 
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A framework  that  describes  the  reasons  for  individual  choices  and  preferences  in  a particular  sociocultural 
context  is  Douglas’  Typology  of  Grid  and  Group.  Douglas  (1982)  posits  that  the  identification  of  grid  (the  roles  and 
rules  that  define  members  of  an  organization)  and  group  (the  social  experience  of  the  members)  shape  the 
perceptions  of  those  within  the  organizational  group. 

Mary  Douglas’  grid  and  group  analysis  (1982)  provides  a framework  for  understanding  underlying 
processes  of  social  change  and  addressing  two  “central  and  eternal  questions  of  human  existence:  ‘who  am  I?’  and 
‘how  should  I behave?”’  (Schwarz  & Thompson,  1990,  p.  6).  Douglas  espoused  the  use  of  grid  and  group  “for 
anyone  desirous  of  checking  out  the  pressures  of  constraint  and  opportunity  which  are  presumed  to  shape  individual 
response  to  the  social  environment”  (Gross  and  Rayner,  1985,  p.  xxii).  They  are  the  two  factors  which  contribute  to 
social  constraints  in  complex  interactions  between  individuals  within  organizations  and  the  organization’s 
environment.  Grid  is  the  dimension  of  individuation  of  members  of  the  organization,  and  group  is  the  dimension  of 
social  incorporation  of  members  in  the  organization.  These  two  dimensions  serve  as  a screen  through  which  the 
culture  of  an  organization  allows  options  to  be  perceived.  (Douglas,  1982).  Assessing  the  relative  strength  of  these 
dimensions  is  a valuable  tool  in  understanding  the  values  and  belief  dimensions  that  are  characteristic  of  a specific 
context. 

Grid  represents  the  degree  to  which  an  individual’s  choices  are  constrained  within  a social  system  by 
imposed  formal  prescriptions  such  as  role  differentiation,  rules,  and  expectations  (Douglas,  1982).  A high/low 
continuum  can  show  grid  strength.  High  grid  refers  to  a social  context  in  which  an  explicit  set  of  institutional 
classifications  regulate  individual  interactions  and  restrain  their  autonomy , or,  in  other  words,  role  and  rule  dominate 
social  interactions  (Douglas,  1982).  Moving  down  the  continuum,  roles  become  more  achieved  than  ascribed,  and 
individuals  are  increasingly  expected  to  negotiate  their  own  relationships  and  life  choices.  At  the  low  end  of  the  grid 
continuum,  there  are  few  distinctions  among  members;  individuals  are  esteemed  more  for  their  behavior  or  character 
than  their  role  status.  Four  criteria  are  used  by  Douglas  (1982)  to  determine  grid:  insulation,  autonomy,  control,  and 
competition. 

Gross  and  Rayner  (1985)  suggested  that  group  represents  the  degree  to  which  people  value  collective 
relationships  and  are  committed  to  a social  unit  larger  than  the  individual.  Evaluation  of  group  involves  recognition 
of  the  holistic  aspect  of  social  incorporation  and  the  extent  to  which  people’s  lives  are  absorbed  and  sustained  by 
corporate  membership  (Harris,  1995).  A low/high  continuum  shows  group  level.  On  the  high  end  of  the  continuum, 
there  are  specific  membership  criteria  and  explicit  pressures  to  consider  group  relationships.  The  survival  of  the 
group  is  more  important  than  the  survival  of  individual  members  within  it,  perpetuating  the  life  of  the  social 
collective  rather  than  its  individual  members.  In  a low  group  social  context,  people  are  not  constrained  by  or  reliant 
upon  a group  of  others,  and  they  experience  more  of  a competitive,  entrepreneurial  way  of  life  (Gross  and  Rayner, 
1985).  Douglas’  four  criteria  to  evaluate  group  included  survival/perpetuation,  membership  criteria,  life  support,  and 
group  allegiance. 

When  simultaneously  considering  high  or  low  strength  in  both  the  grid  and  group  dimensions,  Douglas’ 
four  distinct  possibilities  of  social  environments  emerge  as  presented  in  Figure  1 . 
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FIGURE  1.  Mary  Douglas’  Typology  of  Social  Environment  Prototypes 

Since  its  introduction,  grid/group  analysis  has  undergone  considerable  theoretical  elaboration  (Douglas, 
1982,  1989,  1992;  Douglas  & Wildavsky,  1982;  Thompson,  Ellis,  & Wildavsky,  1990).  Researchers  inspired  by 
Douglas’  insights  have  used  the  framework  primarily  for  describing  particular  social  units  and  constructs  such  as 
technology  policy  (Schwarz  & Thompson,  1990),  high-tech  firms  (Caulkins,  1997),  work  cultures  (Mars  & Nicod, 
1984),  career  expectations  (Hendry,  1999),  higher  education  (Lingenfelter,  1992),  and  school  culture  (Harris,  1995). 

Methodology 

The  participants  in  this  explanatory  case  study  (Yin,  1993)  included  higher  education  faculty  members 
within  the  College  of  Veterinary  Studies  (CVS)  and  the  College  of  Human  Ecology  (CHE)  at  a four-year,  land  grant 
research  institution,  which  will  be  referred  to  as  Midwestern  University. 

Purposive  sampling  strategies  were  applied  to  the  selection  of  colleges  and  respondents.  The  two  colleges 
selected  consisted  of  faculty  members  who  have  exhibited  use  of  IT  to  varying  degrees  in  their  teaching  activities.  A 
minimum  of  one  academic  administrator,  one  IT  staff  member,  three  faculty  members  who  use  IT,  and  three  faculty 
members  who  do  not  use  IT  were  initially  interviewed,  and  further  interviews  were  conducted  as  a result  of  member 
referral. 

Data  were  collected  through  questionnaires,  interviews,  document  and  artifact  analysis,  and  participant 
observation,  with  the  aim  of  corroborating  emergent  facts  or  phenomena.  As  participants  answered  the 
questionnaire,  their  results  were  forwarded  to  the  investigator  via  email  and  stored  electronically  in  a spreadsheet. 
Eor  analysis  of  items,  a frequency  count  was  conducted.  Analysis  of  the  interview  responses  was  done  from 
verbatim  transcriptions.  The  field  notes  taken  during  and  immediately  following  observations  were  transcribed  and 
analyzed  along  with  the  interviews.  Analysis  of  documents  and  artifacts  took  place  as  they  were  gathered.  All 
analysis  activities  occurred  simultaneously  with  data  collection  and  were  ongoing  throughout  the  study.  As  data 
were  collected,  physical  organization  and  filing  was  conducted  using  the  qualitative  software  N*VIVO.  The 
Douglas  typology  was  used  in  classifying  the  data. 

Presentation  of  Cases 

Midwestern  University  is  a modern,  comprehensive  land  grant  university,  which  strives  to  serve  the  state, 
national,  and  international  communities  by  providing  its  students  with  exceptional  academic  experiences  and  by 
conducting  scholarly  research  and  other  creative  activit  ies  that  advance  fundamental  knowledge.  Approximately 
26,000  students  are  enrolled  on  four  campuses.  Undergraduate,  Masters'  and  Doctoral  degrees  are  offered  in  a large 
number  of  fields.  The  university  claims  to  be  emerging  as  a leader  in  network  computing  resources  and  is  accredited 
by  a regional  association  of  colleges  and  schools. 

Case  One:  The  College  of  Human  Ecology 

The  College  of  Human  Ecology  (CHE)  is  housed  in  two  redbrick  buildings  conjoined  by  a second-floor 
atrium/walkway.  One  building  contains  offices  for  the  college  administration,  general  classrooms,  faculty  offices  for 
three  of  the  four  academic  departments  in  the  college,  and  offices  for  college  service  units.  The  three  academic 
departments  located  in  this  building  are  Eashion  and  Interior  Design,  Nutrition  and  Dietetics,  and  Child/Eamily 
Relations.  Offices  range  in  size  and  appearance  from  large,  recently  remodeled  administrative  suites  for  the  Dean, 
Associate  Dean,  and  Assistant  Dean  and  their  respective  administrative  assistants,  to  faculty  offices  that  are 
approximately  one-fourth  the  size  of  the  smallest  administrative  office  with  unadorned  walls,  a bare  floor,  and  old 
furniture.  Office  space  is  grouped  according  to  discipline  and  department,  with  classrooms  interspersed  throughout. 
The  classrooms  are  primarily  traditional  in  construct  and  hold  an  average  of  30  wooden  chairs  with  attached  desks,  a 
whiteboard  at  the  front  of  the  room,  an  instructor’s  podium,  a pull-down  projection  screen,  and  an  overhead 
projector  on  a rolling  cart. 

Two  classrooms  in  the  college  have  permanently  installed  multimedia  presentation  stations  with  a touch- 
pad  control  panel,  a computer,  a VCR,  and  laserdisc  player.  These  classrooms  are  larger  than  the  others,  seating 
approximately  60  students  in  rows  of  chair/desk  combination  seating.  Eaculty  can,  and  do,  request  to  teach  in  these 
two  classrooms.  One  respondent  said  that  the  college  had  been  “chastised  for  using  Student  Tech  Eee  money  for 
purchasing  equipment  for  faculty  to  teach  with  in  classrooms.’’  The  Student  Technology  Eee  is  a per-credit-hour  fee 
charged  each  Midwestern  University  student.  A portion  of  the  money  goes  to  each  college  to  facilitate  the  purchase 
of  technology  expenditures  benefiting  students  directly. 
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Grid  Considerations  from  Douglas’  Typology 

CHE  is  categorized  as  a high  grid  environment  for  many  reasons.  Property  is  viewed  as  a status  symbol, 
which  is  typical  of  a high  grid  culture.  Work  related  tools  in  a high-grid  environment  are  allotted  to  faculty  by  the 
college  or  unit  administration.  The  Associate  Dean  and  the  Dean  in  CHE  both  held  responsibility  for  decisions 
regarding  IT.  The  college-level  technical  support  personnel  were  under  the  direct  supervision  of  the  Associate  Dean. 
IT  tools  for  faculty  members  were  initially  purchased  through  a college-level  fund  with  the  Dean’s  approval. 

All  spaces  were  maintained  at  the  college  level,  indicating  a typical  high  grid  environment  in  which  higher 
positions  have  more  control  over  property.  The  three  college  administrators  - the  Dean,  Associate  Dean,  and 
Assistant  Dean  - officed  in  separate,  much  nicer  spaces  than  faculty  members.  The  phrases,  “first-floor  mentality” 
and  “first-floor  agenda,”  indicating  the  tangible  separation,  were  voiced  many  times  during  the  interviews.  Of  the  20 
general  instructional  spaces  in  the  college,  only  two  were  equipped  with  multimedia  presentation  equipment.  During 
renovation  of  classrooms,  one  faculty  member  had  recommended  to  administration  that  they  consider  “what 
classrooms  [would]  really  be  like  if  we  actually  start  using  technology  not  just  as  a delivery  means,  like  PowerPoint 
or  something,  if  it’s  actually  interactive.”  The  administration  did  not  take  his  suggestion,  as  evidenced  by  classrooms 
arranged  in  rows  with  traditional  desk/chair  combinations  and  minimal  options  for  teaching  presentation  beyond 
stand-up  lecture  delivery. 

A typical  high-grid,  centralized,  dominant  authority  structure  was  prevalent  in  CHE.  Repeatedly,  the  Dean 
of  the  college  was  described  as  “strong”  and  “very  much  in  control.”  A “top-down  control”  was  emphasized 
throughout  the  college.  A high  grid  environment  also  typically  features  an  administration  consciously  involved  in 
maintaining  setting  and  maintaining  rules  through  the  hierarchical  roles  that  exist.  The  vision  for  the  CHE  was 
clearly  “the  Dean’s”  vision,  and  the  expectation  that  faculty  members  would  conform  to  this  vision  was  just  as  clear. 
One  faculty  member  explained  that  while  the  Dean  “makes  a sincere  effort”  to  get  input  from  faculty  members  when 
developing  vision  for  the  college,  it  doesn’t  actually  happen.  There  was  no  indication  of  faculty  members’ 
perceptions  and  ideas  being  valued  by  administration.  Rather,  the  division  between  administration  and  faculty 
members  was  pervasive. 

This  gap  was  particularly  noticeable  regarding  IT  use.  While  the  administration-directed  goal  of  the  college 
being  recognized  as  “state-of-the-art”  in  IT  use  was  widely  expressed,  the  Dean’s  reputation  for  ineptness  with  IT 
and  unwillingness  to  use  IT  personally  was  just  as  widely  expressed.  The  administration  encouraged  faculty 
members  to  attend  training  and  made  that  investment  in  the  faculty  members  who  were  willing  to  attend  training, 
which  was  typically  conducted  by  experts  outside  of  the  college. 

Each  faculty  member  begins  as  an  Assistant  Professor  and  has  the  opportunity  for  achieving  promotion  and 
tenure  through  the  guidelines  set  forth  in  the  department  level  and  college-level  Retention,  Promotion,  and  Tenure 
(RPT)  documents.  While  the  fact  that  individuals  can  achieve  roles  is  a low  grid  characteristic,  the  rules  associated 
with  the  roles,  which  are  created  at  the  department  and  college  levels,  indicate  a high-grid  culture.  The  pursuit  of 
RPT  comes  with  certain  risks.  Committees  at  the  department  and  college  level  determine  the  success  of  individual 
faculty  members’  adherence  to  the  guidelines.  If  the  faculty  member  is  not  successful,  he  or  she  risks  not  being 
reappointed,  tenured,  or  promoted.  This  process  is  indicative  of  a high-grid  environment,  in  which  authorities  and 
experts  make  the  decis  ion  to  set  risks  at  a certain  level  and  set  standards  to  mitigate  risks  (Gross  & Rayner,  1985). 

Since  IT  use  was  not  specifically  outlined  in  RPT  documents,  many  faculty  members  perceived  that  it  was 
a risk  to  spend  time  developing  and  using  IT.  The  motivation  to  use  IT  was  overwhelmingly  self-defined  for  CHE 
faculty  members.  “Enhancing  instruction,”  “enriching  students’  experiences,”  and  “keeping  up  with  the  rest  of  the 
world”  were  the  primary  motivations  reported  by  faculty  members.  Without  RPT  as  a motivation  to  use  IT,  the  CHE 
administration  encouraged  IT  use  by  providing  opportunities  for  faculty  members  to  obtain  special  training  in  this 
area,  but  they  were  expected  to  complete  all  development  and  delivery  tasks  themselves.  When  they  needed 
assistance,  faculty  members  sought  support  from  individuals  outside  the  college  rather  than  using  internal  resources. 
When  faculty  members  took  advantage  of  offered  training  opportunities,  they  were  expected  to  produce  a return  on 
the  college’s  investment.  One  faculty  member  in  particular  described  an  incident  in  which  she  accepted  one  month’s 
salary  as  an  incentive  for  initial  development  of  an  online  course.  After  teaching  it  once,  she  had  decided  not  to  do  it 
again.  The  department  head,  however,  reminded  her  they  had  made  an  investment  - the  initial  one-month’s  salary - 
and  expected  her  to  teach  the  online  course  once  per  year  with  no  further  reward. 

Group  Considerations  from  Douglas’  Typology 

In  a high  group  culture,  ownership  and  management  of  property  and  resources  are  organized  for  the  benefit 
of  the  whole  group  (Lingenfelter,  1996).  Initial  purchasing  of  IT  tools  in  CHE  was  done  at  the  college  level,  and 
technical  support  for  these  tools  was  organized  as  one  college-level  group.  Equitability  of  desktop  resources  for 
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faculty  members  was  expressed  as  a goal,  and  the  Associate  Dean  described  the  centralization  of  IT  purchasing  and 
service  as  a means  to  this  end. 

Both  physical  spaces  and  cyber  spaces  were  maintained  at  a college  level.  The  website  for  the  college  was 
built  and  maintained  by  a centralized  person,  and  was  “updated  at  the  discretion  of  the  administration.’’  One  faculty 
member  in  particular  expressed  frustration  that  “the  determination  was  made  at  the  college  level  what  elements 
would  be  on  the  faculty  web  pages  and  what  format  would  be  used.’’  This  is  a strong  indicator  of  a high  group 
environment  in  which  communication  flows  primarily  through  corporately  regulated  or  maintained  processes 
(Lingenfelter,  1996). 

Within  a high-group  environment,  work  can  be  described  as  corporately  organized  cooperation 
(Lingenfelter,  1996).  While  individuals  in  this  type  of  environment  have  separate  work  activities,  the  group  may  call 
on  members  to  participate  in  corporately  organized  activities.  This  description  fits  CHE  well.  Individual  faculty 
members  perform  separate  work  activities  in  accordance  with  his  or  her  personal  interests  and  professional 
discipline.  Department-level  and  program-level  groups  plan  schedules  and  details  of  courses  to  be  offered  within 
their  disciplines.  Assignments  are  made  to  faculty  members  within  the  groups  to  teach  certain  courses. 

Another  high-group  example  is  CHE’s  involvement  as  a partner  in  a consortium  of  eight  other  colleges  in 
the  midwestern  section  of  the  United  States.  The  mission  of  the  consortium  is  to  create  a collaborative  environment 
where  students  can  take  graduate  level  courses  at  a distance  from  any  of  the  participating  universities,  and  CHE’s 
contribution  is  to  eventually  offer  two  degree  programs  completely  online.  Individual  faculty  members  are  expected 
to  contribute  to  this  effort  by  teaching  an  online  course  when  they  are  asked  to. 

In  a high-group  environment,  the  institution  decides  which  risks  are  socially  acceptable  and  which  are  not 
(Gross  & Rayner,  1985).  Reappointment  and  tenure  expectations  in  CHE  were  outlined  by  department-level  RPT 
documents  and  directed  by  department  heads.  In  addition,  faculty  members  faced  college-level  RPT  documents  and 
college-level  administrative  approval. 

An  indication  of  a low-group  environment  emerged  in  CHE  along  with  the  perception  of  risk  in  relation  to 
IT  use  and  RPT.  The  risk  of  spending  time  developing  and  using  IT  was  perceived  as  higher  for  faculty  members 
who  were  not  yet  tenured,  since  IT  use  was  not  recognized  in  the  RPT  process  and  non -tenured  faculty  members’ 
activities  were  more  tightly  bound  by  RPT  guidelines. 

IT  use  within  CHE  was  initiated  and  planned  by  individual  faculty  rather  than  the  whole  group,  indicating  a 
low-group  situation.  Individuals  were  not  assigned  tasks  and  responsibilities  related  to  IT  use  unless  an  investment 
of  training  had  been  made  in  them.  IT  use  was  not  standardized  across  the  college,  but  one  software  package  for 
electronic  presentation  (PowerPoint)  and  another  for  Internet  course  delivery  (LearningSpace)  were  used  by  an 
overwhelming  majority  of  people  who  participated  in  those  activities.  The  college  had  offered  extensive  training  on 
these  two  software  packages.  Two  different  departments  spoke  of  specialized  software  they  use  in  their  respective 
programs  that  the  other  departments  in  the  college  have  no  use  for. 

In  a larger  sense  of  the  concept  of  time,  this  college  seemed  to  reflect  a strong  desire  to  keep  a firm  grip  on 
the  future.  On  the  college  website,  the  term  “future -oriented’’  is  used  repeatedly  in  describing  CHE.  In  interviews, 
faculty  members,  administrators,  and  staff  cited  the  desire  to  “keep  up  with  the  future’’  as  a reason  why  they  use  IT. 
Juxtaposed  with  this  apparent  grasp  for  the  future  and  its  innovations,  is  the  strong  history  and  practice  of  deeply 
rooted  higher  education  traditions  such  as  RPT  policies  and  an  organizational  structure  built  upon  the  position  of  a 
“strong  Dean.’’ 

Case  Two:  The  College  of  Veterinary  Studies 

The  College  of  Veterinary  Studies  (CVS)  is  housed  in  a three-building  complex  on  the  outskirts  of  the  main 
Midwestern  University  campus.  The  oldest  building,  McDonald  Hall,  serves  as  the  main  instructional  area  for  first- 
year,  second-year,  and  third-year  students,  with  three  theater-style  classrooms  for  each  of  the  respective  classes, 
smaller  teaching  laboratories,  and  a computer  lab.  The  college’s  library.  Student  Resource  Center,  the  Office  of 
Multimedia  Curriculum  Development,  and  most  of  the  administrative  offices  are  housed  in  this  building.  Two  of  the 
college’s  three  academic  departments,  the  Department  of  Veterinary  Pathobiology  and  the  Department  of 
Physiological  Sciences,  have  faculty  and  department  head  offices  in  McDonald  Hall 

The  Veterinary  Medical  Teaching  Hospital  is  the  largest  of  the  three  buildings  in  the  CVS  complex,  and 
offers  full  veterinary  services  to  the  public.  In  addition  to  the  for-profit  services,  the  teaching  hospital  is  also  home 
to  administrative  and  faculty  offices  and  instructional  spaces  for  the  third  academic  department  in  the  college,  the 
Department  of  Veterinary  Clinical  Sciences. 
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Grid  Considerations  from  Douglas’  Typology 

From  a low^grid  perspective,  property  is  typically  viewed  as  a means  to  an  end  (Lingenfelter,  1996).  CVS 
has  focused  on  creating  a pervasive  environment  for  the  ultimate  goal  of  “molding  a good  veterinarian.”  The  goal  of 
providing  a pervasive  environment  to  meet  this  end  extends  to  IT  use  as  well.  Individual  faculty  members  who  elect 
to  teach  in  one  of  the  four  traditional,  lecture-style  classrooms  have  multimedia  presentation  equipment  in  each. 
Smaller  classrooms  and  labs  had  been  set  up  for  faculty  member  who  chose  to  teach  in  a small-group,  more  learner- 
centered  manner.  For  faculty  members  who  opted  for  delivering  web-assisted  courses,  an  Office  of  Multimed ia 
Curriculum  Development,  a centralized  development  service  for  college-wide  website  resources,  provided  templates 
designed  for  the  case-based  curriculum  approach  adopted  by  the  college.  In  addition,  instructional  technology  staff 
personnel  were  available  to  input  the  faculty  members’  content  to  the  website  templates.  Since  the  faculty  members 
individually  chose  which  services  and  opportunities  to  take  advantage  of,  this  environment  is  considered  low-grid. 

In  contrast  to  the  low-grid  factors  considered  above,  two  aspects  of  CVS  were  more  naturally  placed  high 
on  the  grid  continuum:  1)  desktop  computing  resources  were  allotted  to  faculty  members  by  the  College  or  unit 
administration,  and  2)  the  Office  of  Multimedia  Curriculum  Development  was  described  as  providing  “grunt  work” 
for  faculty  members. 

Since  administrators  and  faculty  members  were  dedicated  to  the  common  mission  of  “molding  a 
veterinarian,”  role  distinction  seemed  to  fit  best  on  the  lower  end  of  the  grid  continuum.  The  college  authority 
structures  were  centralized  - a high-grid  notion  - yet  administrators  were  often  described  as  “serving  in  the  trenches 
with  fellow  faculty  members.”  Their  desire  and  efforts  to  participate  in  the  same  activities  as  the  faculty  members  in 
the  college  indicates  a low-grid  environment. 

The  Dean  of  the  college  noted  in  a local  newspaper  article  his  feeling  of  being  “blessed  with  a group  of 
talented  people.”  This  belief  in  talented  people  is  evident  in  faculty  members’  assignments,  which  are  adjusted  on  a 
percentage  basis  in  accordance  with  individual  talents.  Faculty  members  lend  their  talents  to  team  teaching 
endeavors  in  CVS,  where  teaching  teams  were  comprised  of  both  faculty  members  who  held  D.V.M.  degrees  and 
those  who  held  Ph.D.s.  Some  distinction  between  these  two  groups  was  evident  in  interviews,  but  it  was  clear  the 
differences  in  individual  skills  were  valued.  This  situation  is  indicative  of  a low-grid,  task/goal  system  in  which 
individual  assignments  change  in  accordance  with  the  goal  or  need  at  a specific  time  (Lingenfelter,  1996). 

Students  were  also  presented  with  the  choice  to  take  advantage  of  resources  and  services.  The  curriculum 
was  designed  to  offer  a variety  of  opportunities  for  students,  an  indication  of  a low-group  environment.  Due  to  a 
high  student-to-faculty  member  ratio  and  keeping  each  class  intact  throughout  the  four-year  program,  CVS  students 
seemed  to  play  a very  different  role  than  students  do  in  other  colleges.  Faculty  members  and  students  have  a great 
deal  of  close  contact,  particularly  in  the  fourth  year,  when  students  and  faculty  members  are  working  side-by-side  in 
the  teaching  hospital.  Many  faculty  members  mentioned  that  student  expectations  for  the  use  of  IT  were  very 
significant.  In  addition,  student  evaluations  of  courses  carried  a great  deal  of  weight  in  this  college.  The  Associate 
Dean  for  Academic  Affairs  described  an  incident  in  which  a faculty  member’s  teaching  assignment  was  lessened 
due  to  poor  student  evaluations.  The  strong  voice  that  students  have  in  this  college  is  indicative  of  a low-grid 
environment. 

Indicative  of  a low -grid  environment,  faculty  members  within  CVS  achieve  rank  and  role  based  on 
personal  productivity  within  their  individual,  percentage-based  assignment.  Documents  for  reappointment, 
promotion,  and  tenure  (RPT)  provide  guidelines  for  this  process.  Committees  at  the  department  and  college  level 
determine  the  success  of  individual  faculty  members’  adherence  to  the  guidelines.  If  the  faculty  member  is  not 
successful,  he  or  she  risks  not  being  reappointed,  tenured,  or  promoted.  This  process  is  indicative  of  a high-grid 
environment,  in  which  authorities  and  experts  make  the  decision  to  set  risks  at  a certain  level  and  set  standards  to 
mitigate  risks  (Gross  & Rayner,  1985).  Yet,  CVS  faculty  members  are  rewarded  through  the  RPT  process  based  on 
their  individual  percentage  assignment,  a low-grid  characteristic. 

Since  IT  use  was  not  specifically  outlined  in  RPT  documents,  many  faculty  members  perceived  that  it  was 
a risk  to  spend  time  developing  and  using  IT.  In  particular,  the  faculty  members  who  had  a higher  percentage 
assignment  in  research  reported  the  necessity  of  avoiding  IT  use  in  order  to  successfully  meet  RPT  guidelines. 

While  curricular  decisions  were  reportedly  prescribed  by  the  institution  in  CVS  - a high-grid  concept  - 
faculty  members  were  motivated  more  by  self -defined  interests,  a low-grid  concept.  Faculty  members  reported  using 
IT  primarily  to  “enhance  learning,”  because  “it’s  fun”  and  “easy  and  effective.”  In  addition,  they  cited  pressure  from 
other  veterinary  schools,  animal  producers,  other  faculty  members,  and  society  as  motivators.  The  typical  IT  user  in 
CVS  was  described  as  someone  who  “thinks  outside  the  traditional  box”  and  is  “not  a performer,  they’re  a player, 
they’re  a participant.”  These  motivations  represented  intrinsic  factors  that  outweighed  the  absence  of  extrinsic 
rewards  for  IT  use  from  the  college. 
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In  addition  to  intrinsic  motivations,  personal  barriers  to  IT  use  also  emerged  in  discussions  with  CVS 
faculty  members.  Most  significantly,  faculty  members  expressed  that  lack  of  time  was  the  biggest  barrier  to  IT  use. 
With  a strong  recognition  of  the  amount  of  time  needed  to  invest  in  using  IT  successfully,  faculty  members  noted 
that  spending  that  time  had  to  be  balanced  with  the  expected  benefit  and  other  scholarly  pursuits.  Personal  issues 
such  as  intimidation,  apprehension  regarding  assistance,  and  being  uninformed  were  also  cited  as  barriers. 

Group  Considerations  from  Douglas’  Typology 

In  CVS,  work -related  tools  were  purchased,  regulated,  and  maintained  by  the  college,  indicating  a high- 
group  environment,  which  features  corporate  ownership  of  property.  Desktop  computers  and  printers  for  faculty 
offices,  multimedia  presentation  tools  in  classrooms,  and  IT  tools  in  the  Office  of  Multimedia  Curriculum 
Development  were  purchased  with  college-level  finances  and  supported  by  the  college-wide  technology  support 
personnel. 

When  faculty  members  do  have  questions  or  need  assistance  regarding  IT  use,  they  have  the  Office  of 
Multimedia  Curriculum  Development  staff  and  tools  available.  Students  were  able  to  take  advantage  of  this  service 
as  well.  The  office  also  handled  the  development  and  maintenance  of  an  online  database  of  digital  images.  Faculty 
members  voluntarily  contributed  images  they  had  personally  gathered  to  add  to  this  college-wide  database.  This 
service  and  facility  is  a high-group  example  of  a society  that  organizes  itself  to  manage  its  resources  for  the  benefit 
of  the  whole. 

Another  indicator  of  a Corporate,  high-group  environment  is  that  the  organization  has  a lifespan  that 
exceeds  the  life  of  individuals  in  it  (Lingenfelter,  1996).  CVS’s  group  mission  of  “molding  veterinarians’’  transcends 
any  individual  effort  or  interest.  Professional  clinic  services  through  the  teaching  hospital  serve  the  common  mission 
throughout  the  college  of  “molding  veterinarians.’’  In  a continued  effort  to  provide  a pervasive  IT  environment,  the 
teaching  hospital  was  designed  so  students  could  “experience  different  technologies  that  they  may  be  using  as  a 
practitioner.” 

The  relationship  between  individuals  in  different  roles  was  described  as  creating  “a  family  environment”  by 
one  faculty  member  and  as  “a  unique,  close-knit  environment”  by  another.  These  descriptions  indicate  a high-group 
environment,  in  which  the  entire  group  shares  a focus  and  corporate  labor  involves  extensive  social  interaction 
among  the  participants  (Lingenfelter,  1996).  Authority  within  the  college  is  organized  corporately,  with  clear 
accountability  for  individual  responsibilities.  The  administration  clearly  saw  its  role  as  one  of  providing  a 
environment  for  faculty  members  to  take  advantage  of  opportunities.  Faculty  members  collaborated  on  teaching 
teams  to  create  opportunities  for  students  within  the  curriculum.  Students  and  staff  voices  were  heard  and  expertise 
is  valued  as  well.  Lingenfelter  (1996)  described  work  in  a high-group  environment  as  “corporately  organized 
cooperation.”  This  description  is  fitting  of  CVS. 

In  a high-group  environment,  the  institution  decides  which  risks  are  socially  acceptable  and  which  are  not 
(Gross  & Rayner,  1985).  RPT  expectations  in  CVS  were  outlined  by  department-level  RPT  documents  and  directed 
by  department  heads.  In  addition,  faculty  members  faced  college-level  RPT  documents  and  college-level 
administrative  approval.  The  risk  of  spending  time  developing  and  using  IT  was  perceived  as  higher  for  faculty 
members  who  had  a lower  percentage  assignment  for  teaching  than  research. 

Indications  of  a low-group  environment  also  emerged  in  CVS.  Different  departments  in  the  college  had 
different  RPT  documents.  In  addition,  individual  faculty  members  did  initiate  and  plan  their  work  and  labor 
activities. 

Findings 

The  colleges  studied  offer  two  different  cultural  representations  of  the  Douglas  (1982)  typology.  CHE 
presented  a high-grid,  high-group,  or  "Corporate”  playing  field,  in  which  administrative  roles  were  at  the  top  of  the 
hierarchical  structure.  The  high  degree  of  administrative  control  over  and  highly  regulated  processes  of 
procurement,  allotment,  and  use  of  property  created  an  environment  in  which  individual  choice  was  restricted  only 
to  what  was  made  available  by  decision  makers.  Negotiation  for  resources  and  opportunities  related  to  IT  was  not  a 
common  practice.  In  terms  of  IT  use,  very  few  tools  were  readily  available,  and  technical  and  instmctional 
assistance  outside  of  the  college  was  sought  and  relied  upon.  Faculty  members  perceived  the  use  of  IT  as  high  risk 
for  someone  who  had  not  yet  been  promoted  or  tenured.  With  four  diverse  departments,  this  disparate  social  context 
contributed  to  individual  roles  being  highly  insulated  from  others. 

In  CVS's  Collectivist  context  (low-grid,  high-group),  the  effort  to  create  a “pervasive  environment”  for  IT 
use  was  paramount  and  evidenced  by  IT  tools  in  all  classrooms  and  a well -funded  and  well-trained  technical  and  IT 
staff.  In  addition,  all  members  of  the  college  were  united  by  one  mission  and  one  purpose  - to  “mold  a good 
veterinarian.”  Faculty  members  as  well  as  students  were  expected  to  make  individual  choices  regarding  use  of 
available  resources.  Administration  was  perceived  as  serving  “in  the  trenches  with  fellow  faculty  members”  and  as 
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models  for  instructional  practices.  Any  risk  associated  with  IT  use  was  low  for  faculty  members  whose  percentage- 
based  assignment  was  heavily  weighted  toward  teaching,  but  high  for  those  with  a high  percentage  research 
assignment.  Decisions  regarding  procurement,  allotment,  and  use  of  property  were  more  closely  tied  to  the  shared 
mission  than  to  administrative  control.  Faculty  members’  choices  to  use  IT  were  less  restricted  and  less  regulated, 
evidenced  by  open  and  welcomed  negotiation  for  opportunities  and  resources  related  to  IT. 

Conclusions 

The  findings  of  these  two  cases  indicate  that  barriers  and  incentives  to  IT  use  are  more  closely  aligned  with 
the  cultural  context  in  CVS  than  they  are  in  CHE.  In  the  lo\wgrid,  high-group  environment  of  CVS,  IT  use  appeared 
to  be  further  developed.  The  Office  of  Multimedia  Curriculum  Development,  the  case-based  approach  to  curriculum, 
modeling  of  IT  use  by  administrators  and  curriculum  leaders,  and  the  availability  of  internal  IT  experts  appeared  to 
facilitate  a higher  level  of  IT  use  in  CVS.  Students  were  also  involved  in  developing  and  presenting  IT-based 
products  and  were  offered  assistance  and  tools  for  doing  so.  While  the  relationship  between  Grid  and  Group 
Typology  and  faculty  members  IT  use  appears  to  be  useful,  a predictive  element  to  the  relationship  could  not  be 
established  within  the  parameters  of  this  research  study,  since  barriers  to  and  incentives  for  IT  use  did  not  fit  neatly 
within  the  grid/group  typology. 

It  is  significant  to  note  that  the  cultural  bias  of  many  of  the  faculty  in  CHE  is  individualistic  in  perceptions 
of  incentives  to  use  IT.  This  individualistic  cultural  bias  is  juxtaposed  against  the  overall  cultural  identity,  which  is 
Corporate,  or  Hierarchical.  This  can  be  one  explanation  for  apparent  discrepancies,  conflict,  and  dissatisfaction 
among  the  members  of  the  college.  In  contrast,  the  cultural  bias  of  the  faculty  and  the  cultural  identity  of  the  college 
in  CVS  are  more  in  harmony.  Eor  example,  incentives  are  also  intrinsic  and  self-defined,  but  this  low-grid 
characteristic  is  in  sync  with  the  Collective  environment  of  the  College.  Because  of  this  alignment,  there  are  less 
dis  crepancies,  conflict,  and  dissatisfaction. 

The  findings  from  this  case  study  impacted  theory,  research,  and  practice.  The  significance  of  using 
Douglas’  (1982)  Grid  and  Group  Typology  as  the  theoretical  framework  in  this  study  lends  credence  to  research 
calling  for  a cultural  perspective  of  IT  use.  Goodman,  Griffith,  and  Tenner’s  (1990)  research  supported  the  notion 
that  a question  such  as  “Why  do  people  decide  to  use  or  not  to  use  the  enabling  technology?’’  (p.  75)  must  be 
explored  with  an  examination  of  the  complicated  relationships  among  technology,  the  individual,  and  the 
organization.  Hagner  (2000)  found,  “If  institutional  culture  is  an  important  consideration  affecting  the  success  or 
failure  of  teaching  transformation,  innovators  must  consider  the  systemic  characteristics  rather  than  the  “practice’’ 
characteristics  prior  to  transformation’’  (p.  32).  Sproull  and  Goodman  (1990)  found  that  changes  in  technology  or 
use  of  technology  lead  to  changes  in  the  social  structure  of  an  organization.  In  turn,  changes  in  the  organization’s 
social  structure  enacted  further  change  on  the  technology.  The  implication  of  this  reciprocal  causation  bears  on  a 
research  design  capable  of  capturing  “the  dynamic  relationship  between  technology  and  the  social  construction  of 
that  technology’’  (p.  259). 

Douglas’  (1982)  Grid  and  Group  Typology  made  two  primary  assumptions:  1)  that  an  individual  will  fail  to 
make  any  sense  of  his  surroundings  unless  he  can  find  some  principles  to  guide  him  to  behave  in  the  sanctioned 
ways  and  be  used  forjudging  others  and  justifying  himself  to  others,  and  2)  that  the  social  context  of  an  organization 
serves  to  permit  and  constrain  effects  upon  individuals’  choices  (Douglas,  1982,  p.  190).  In  accordance  with  these 
assumptions,  Douglas’  framework  was  useful  as  a descriptive  tool  focusing  on  higher  education  faculty  members’ 

IT  use  preferences.  Its  effectiveness  in  identifying  the  cultural  context  of  two  colleges  assisted  in  examining  the 
relationship  between  cultural  context  and  barriers  and  incentives  related  to  IT  use. 

In  this  study,  the  typology  was  useful  in  focusing  on  cultural  context  issues  while  allowing  the  issues 
related  to  why  faculty  members  choose  to  use  IT  to  emerge.  Throughout  the  study,  the  concepts  of  leadership,  risk, 
labor  issues,  and  res  ources  emerged  as  key  constructs  related  to  IT  use.  One  recommendation  for  further  research 
would  be  to  focus  on  these  constructs  and  their  relationship  to  IT  use  preferences  in  a higher  education  setting 
without  the  lens  of  a theoretical  framework. 

This  study  provided  insights  into  how  and  why  faculty  members  choose  and  are  motivated  to  use  IT.  The 
reviewed  literature  outlined  common  barriers  and  incentives  to  IT  use  in  general,  but  the  theoretical  framework 
helped  put  into  perspective  how  and  why  certain  barriers  and  incentives  operate  in  a given  cultural  context. 

This  study  also  provided  implications  for  practice  related  to  the  nature  of  IT  use  in  higher  education 
settings.  The  findings  of  this  study  indicate  benefits  to  leadership  decisions  related  to  IT  use  in  higher  education 
settings.  The  ability  to  identify  the  cultural  context  of  an  organization  and  its  relationship  to  IT  barriers  and 
incentives  will  allow  the  leader(s)  of  the  organization  to  bring  the  pieces  of  this  puzzle  together  into  a complete 
picture.  Eor  example,  the  College  of  Human  Ecology’s  hierarchical  environment  appeared  to  facilitate  the  reported 
barriers  to  IT  use.  Without  incentives  to  balance  the  barriers,  or  an  adjustment  in  cultural  practices,  IT  use  will  likely 
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remain  in  a static  state  (Haulmark,  2001).  In  addition,  data  collected  revealed  faculty  members’  strong  allegiance  to 
department-level  initiatives  and  issues  juxtaposed  with  a strong,  controlling,  centralized  administration  at  the  college 
level.  Since  the  practice  of  IT  use  is  stilted  by  barriers  related  to  the  current  culture,  it  would  seem  restructuring 
decisions  and  services  related  to  IT  use  to  rest  at  the  departmental  level  would  move  the  culture  down  grid  and  down 
group,  balancing  more  effectively  with  incentives  for  IT  use. 

As  a leader  in  an  educational  institution,  a critical  role  is  to  acclimate  faculty  members  into  the  institution’s 
culture.  While  this  may  appear  to  be  a simple  task,  actually  defining  the  culture  and  providing  for  the  social 
integration  into  this  culture  is  often  an  elusive  process.  This  study  will  assist  leaders  in  realizing  the  necessity  of 
understanding  the  organization’s  cultural  context  and  providing  a method  for  studying  that  context. 

This  study  also  provides  vital  information  to  those  individuals  who  provide  instructional  or  technical 
support  to  higher  education  faculty  members.  Understanding  the  cultural  context  within  which  support  is  offered  is  a 
key  to  effective  service. 

Too  often  we,  as  IT-using  educators,  can  be  found  espousing  the  “importance”  of  technology  tools  without 
even  considering  whether  a particular  tool  has  any  relationship  at  all  to  the  overall  mission  or  purpose,  which 
undoubtedly  should  be  optimizing  the  learning  experiences  of  all  students.  Just  as  often,  non-IT-using  educators  can 
be  found  posturing  against  any  use  of  IT  in  order  to  keep  hidden  the  barriers  they  perceive.  And,  of  course,  all 
educators  can  be  found  using  IT  inappropriately  on  any  given  day  in  any  given  classroom.  In  order  to  understand 
what  is  appropriate,  however,  we  must  first  seek  to  understand  our  own  cultural  context,  and  then  we  can  ask 
ourselves  why  IT  should  be  included  in  our  mission  and  purpose  and  how  we  can  optimize  its  use. 
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Theory  to  Guide  Practice 
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Abstract 

Practitioners  have  enthusiastically,  creatively,  and  sometimes  mi  sinformedly  applied  Edgar  Dale’s  (1946) 
Cone  of  Experience  model  - and  its  various  adaptations  - to  guide  their  work  for  decades  now.  Meanwhile, 
precious  little  has  been  accomplished  by  way  of  examining  and  refining  the  model  (and  its  associated  theories)  to 
give  it  the  concreteness  necessary  for  effectively  guiding  practice.  This  paper  suggests  several  philosophical 
perspectives  by  which  the  gaps  in  the  prevalent  versions  of  Dale's  Cone  could  be  exposed.  It  is  hoped  that  this 
discussion  will  help  start  the  process  of  bringing  about  a revised  theoretical  model  that  is  a more  effective  guide  for 
practitioners. 

Introduction 

Edgar  Dale,  in  his  seminal  (1946)  work  (which  he  revised  subsequently  in  1954  and  1969),  introduced  the 
Cone  of  Experience,  a “visual  aid  to  explain  the  inter-relationships  of  the  various  types  of  audio-visual  materials,  as 
well  as  their  individual  positions  in  the  learning  process”  (p.  37).  Dale  described  the  Cone  as  an  attractive  “visual 
metaphor  of  learning  experiences,  in  which  the  various  kinds  of  audio-visual  materials  appear  in  the  order  of 
increasing  abstraction  as  one  proceeds  from  direct  experience”  (p.  38). 

In  the  final  (1969)  edition  of  his  book  Audiovisual  Methods  in  Teaching,  Dale  introduced  the  Cone  of 
Experience  as  a “pictorial  device”  showing  “the  progression  of  learning  experiences  from  direct,  firsthand 
participation  to  pictorial  representation  and  on  to  purely  abstract,  symbolic  expression”  (p.  108).  Dale  stated  the 
following  specific  attributes  of  the  Cone  device  (1969,  p.  109):  It  indicates  the  broad  base  that  direct  experience 
provides  for  learning  and  communication;  it  classifies  instructional  materials  according  to  the  degree  of  experiential 
concreteness  that  each  provides;  its  various  levels  may  suggest  the  most  appropriate  method  for  teaching  an  abstract 
concept  given  the  child’s  needs  and  abilities  at  the  time;  and  it  suggests  the  interrelated  and  interdependent  nature  of 
learning  experiences  and  instructional  materials. 

In  an  interesting  take  on  the  Cone  model,  Stewart  (1969)  situated  the  concept  of  simulation  within  the 
concrete-abstract  continuum  of  the  Cone,  while  also  simultaneously  explicating  the  related  constructs  of  negative, 
positive,  physical,  and  psychological  siimlation.  From  this  standpoint,  Stewart  proposed  an  S-shaped  relationship 
between  degree  of  physical  simulation  - “the  copying  of  that  part  of  the  real-life  situation  which  is  physically 
involved  in  the  desired  responses  or  behaviors  of  the  learner”  - and  transfer  of  learned  behavior,  stating  that  “up  to  a 
point,  increased  increments  of  transfer  of  learned  behaviors  result  from  increased  degrees  of  physical  simulation”  (p. 
160).  Stewart  also  proposed  that  “if  some  behaviors  are  not  being  performed  adequately  (by  learners  at  a given  level 
of  simulation),  it  is  an  indication  of  a need  for  a greater  degree  of  simulation;”  and  that  “increased  simulation  built 
into  the  course  materials  increase(s)  (learner)  retention.” 

Since  its  inception.  Dale’ s Cone  has  been  a key  feature  in  educational  media  courses  at  universities 
throughout  the  United  States  and  abroad.  Numerous  training  sessions  and  workshops  have  been  held  at  professional 
conferences  and  conventions  in  diverse  fields  to  train  practitioners  in  using  Dale’s  Cone  to  guide  their  work.  Several 
books  and  journal  articles  have  been  written  that  elaborate  on  Dale’s  Cone,  sometimes  making  confused 
interpretations  and  creative  adaptations  of  it,  and  occasionally  advancing  explicit  propositions  regarding  its 
application  to  teaching  and  other  instructional  activities.  All  this  while,  precious  little  has  been  accomplished  by  way 
of  examining  the  Cone  model  (and  its  associated  theories)  to  see  if  it  is  really  meaningful  as  a practitioner’s  guide, 
and  of  refining  it  to  give  it  the  explicit  constraints  and  concreteness  necessary  for  it  to  be  an  effective  guide  of 
practice. 

Subsequent  Usage  of  Dale’s  Cone 

Meanwhile,  very  little  research  seems  to  have  been  accomplished  in  terms  of  elaborating  upon  (and 
empirically  testing)  the  postulates  implied  by  Dale’s  Cone.  It  is  a telling  state  of  affairs  that,  when  this  researcher 
sent  out  an  e-mail  message  to  graduate  students,  scholars,  and  faculty  at  one  of  the  leading  educational  technology 
programs  in  the  country  asking  whether  any  of  the  recipients  had  heard  of  a significant  study  that  had  empirically 
tested  the  postulates  of  Dale’s  Cone,  not  a single  convincing  response  was  received!  On  the  other  hand,  many 
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practitioners  (and  those  that  inform  them)  have  tended  to  unhesitatingly  use  Dale’s  Cone  as  a guide  of  practice, 
without  evidencing  much  thought  regarding  whether  it  was  really  meant  to  be  used  that  way. 

In  an  interesting  case,  this  researcher,  while  attempting  to  find  examples  of  the  empirical  application  of 
Dale's  postulates  in  different  contexts,  contacted  a scholar  who  had  presented  a paper  about  the  application  of  Dale's 
Cone  to  “counseling  in  a university  environment’’  (at  the  2000  National  Conference  of  the  American  Counseling 
Association  in  Washington,  D.C.)  and  requested  a copy  of  the  paper  in  question.  However,  the  scholar  (who  shall 
remain  unnamed)  responded  as  follows:  “I  did  not  publish  the  presentation  that  I did  at  the  conference,  so  I do  not 
have  a paper  to  send  you.  What  I did  was  train  the  university  counselors  on  the  use  of  Dale's  Cone  of  Experience  so 
that  they  could  utilize  it  from  a learning  and  retaining  perspective.  We  focused  on  how  to  integrate  tactile  and  doing 
activities  into  counseling  to  make  things  happen  and  to  encourage  retention  of  new  interpersonal  skills.  My 
presentation  was  on  utilization  and  training  and  was  very  interactive.’’ 

Further  searches  of  the  literature  revealed  journal  articles  such  as  Church  (1999)  and  Kvam  (2000)  that 
made  uncritical  reference  to  Dale’s  Cone  as  a practitioner’ s guide.  Church  (1999)  advised,  “...(W)hen  teaching 
topics  that  involve  human  interaction  with  (a  “system’’  to  perform  some  task),  instructional  design  should  . . . include 
consideration  of  work  by  Dale  (1954)  and  Bruner  (1966).’’  He  went  on  to  describe  Dale’s  and  Bruner’s  theories  and 
states  that  they  are  “related,’’  but  said  nothing  about  whether  either  theory  is  sufficiently  precise  or  concrete  to  be  of 
tangible  use  to  practitioners  that  need  to  make  hard  instructional  design  decisions.  Kvam  (2000)  referred  to  Dale  and 
Bruner  while  advocating  the  use  of  cooperative  learning.  According  to  him.  Dale  (1969)  “suggested  that  learned 
concepts  can  be  more  efficiently  retained  in  one’s  memory  if  more  levels  of  experience  can  be  included  in  the 
learning  process,’’  but  Kvam’s  paper  did  not  evidence  much  concern  regarding  the  need  for  empirical  backing 
behind  such  propositions.  Interestingly,  Kvam’s  article  actually  detailed  the  results  of  an  experimental  study  that  he 
had  conducted  to  “investigate  the  long-term  effects  of  active  learning  methods  on  student  retention.’’  However,  since 
the  study  used  small,  non-random  samples,  the  data  obtained  was  insufficient,  preventing  Kvam  “from  making  any 
strong  statistical  claim’’  that  active  learning  methods  were  more  effective  than  traditional  lecture  methods. 

Meanwhile,  Tipton  (1998)  introduced  the  Cone  as  part  of  a chapter  discussing  the  application  of  technology 
to  learning,  saying  that  Dale’s  and  Bruner’s  theories  “help  us  understand  the  role  of  media  in  the  classroom  in 
relationship  to  learning”  (p.  3).  First  of  all,  it  appears  that  Tipton  got  the  chronological  order  of  these  theories 
incorrect  when  she  said  that  Dale  “took  (Bruner’s)  work  a step  further  with  his  Cone  . . . and  related  (Bruner’s) 
sequence  of  learning  to  mediated  events”  (p.  3),  because  Dale’s  original  (1946)  conception  of  the  Cone  predates 
Bruner’s  theory  by  20  years.  Perhaps  Dale’s  (1969)  use  of  Bruner’s  analysis  to  back  up  his  conception  of  the  Cone 
was  a confounding  factor?  Anyhow,  Tipton  went  on  to  discuss  the  application  of  Dale’s  Cone  to  teaching  in  some 
detail  (p.  5),  proposing  that  a child  needs  the  concrete  experiences  at  the  bottom  of  the  Cone  to  understand  the 
abstract  ones  at  the  top,  and  that  if  all  we  present  is  visual  and  verbal  symbols,  the  student  has  no  concrete  referent 
to  understand  the  meaning.  Once  again,  conspicuously  absent  were  any  attempts  to  empirically  corroborate  any  of 
the  Cone’s  propositions. 

An  Intriguing  Theoretical  Quagmire 

Above  and  beyond,  what  is  even  more  interesting  (and  even  bizarre)  is  that,  for  many  practitioners  and  their 
informants.  Dale’s  Cone  seems  inseparably  joined  to  this  rather  bold  theory  that  suggests  an  explicit  relationship 
between  the  levels  of  Dale’s  Cone  and  learners’  levels  of  recall,  by  saying  that  learners  generally  remember  10  per 
cent  of  what  they  read,  20  per  cent  of  what  they  hear,  30  per  cent  of  what  they  see,  50  per  cent  of  what  they  hear  and 
see,  70  per  cent  of  what  they  say  and  write,  and  90  per  cent  of  what  they  say  as  they  perform  an  authentic  activity.  It 
is  not  clear  who  exactly  came  up  with  these  ideas  (they  have  been  variously  attributed)  or  how  much  these  precise 
figures  are  based  on  hard  empirical  evidence,  but  the  ensuing  confusion  of  these  ideas  with  Dale’ s work  has 
certainly  led  to  a blurring  of  the  boundaries  between  the  two  theories  in  the  minds  of  many  practitioners  and  of  those 
who  inform  them. 

For  example,  Buehler  (2000)  describes  these  six  levels  under  the  title  of  Dale’s  Cone,  as  do  Sjostrom  & 
Forsberg  (2001).  Hall  (2002,  para.  2)  commits  the  same  error,  saying  that  according  to  the  “Edgar  Dale  Cone  of 
Experience,”  a person  remembers  10%  of  what  they  read,  20%  of  what  they  hear,  30%  of  what  they  see,  and  50%  of 
what  they  see  and  hear.  Hall  goes  on  to  explain  that  “the  percentage  increases  for  those  fortunate  enough  to  read, 
hear,  see  and  do  things  in  actual  or  practical  experiences.”  Leggett  (2002,  para.  2)  offers  another  example  of  such  a 
confusion:  “Dale's  Cone  of  Experience  is  a model  used  to  describe  the  degree  of  learning  that  occurs  with  different 
levels  of  abstraction  and  active  participation.  The  farther  you  progress  down  the  Pyramid,  the  greater  the  learning.  It 
has  been  found  that  we  retain  only  5%  of  what  we  hear  but  70%  of  what  we  do.”  Meanwhile,  the  Nottingham  Trent 
University,  Faculty  of  Humanities’  International  Studies  Resource  Centre  Web  site  (ISRC  Online,  2002)  states  that 
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according  to  Dale’s  Cone,  people  remember  20%  of  what  they  hear,  30%  of  what  they  see,  and  50%  of  what  they 
see  and  hear,  and  it  actually  cites  Dale’s  1969  work  as  the  reference  for  this  information! 

Sorenson  (2002,  February)  gets  even  more  creative,  renaming  the  Cone  device  as  the  “Cone  of  Learning’’ 
in  the  following  blithe  advice  to  teachers:  “Passive  activities,  the  kind  of  which  traditional  teaching  is  known  for, 
guarantee  only  a small  percentage  of  retention.  Edgar  Dale’s  “Cone  of  Learning’’  reveals  that  only  10%  of  what 
students  read  is  likely  to  stay  with  them.  20%  of  what  they  hear,  30%  of  what  they  see  is  retained.  While  still  a 
limited  amount,  mixing  the  senses  produces  more  retention.  Up  to  50%  of  what  is  simultaneously  seen  and  heard, 
stays.  Active  participation  ensures  more  learning.  Up  to  seventy  percent  of  what  is  said  remains  with  the  speaker.  Up 
to  ninety  percent  of  what  is  both  said  and  done  is  retained  with  the  presenter.  Therefore,  participating  in  discussions, 
giving  speeches,  doing  dramatic  presentations,  simulating  the  real  thing  and  doing  the  real  thing  ensures  better 
retention.  Anyone  walking  past  a classroom  should  expect  to  see  structured,  mediated  activity!  Never  should  they 
expect  to  see  the  teacher  talking  extensively  beyond  the  first  few  minutes  of  class”  (para.  9). 

There  are  many  more  such  examples  of  the  confounding  of  Dale’ s explication  of  the  Cone  with  these 
mysterious  six  levels , but  the  above  examples  should  suffice  to  make  the  nature  of  this  problem  apparent. 

Pertinent  Issues 

The  main  point  that  this  researcher  is  attempting  to  make  via  this  discussion  is,  it  does  not  appear  that  Dale 
ever  meant  for  his  Cone  to  be  used  by  practitioners  (and  for  that  matter,  by  many  other  theoreticians)  the  way  it  has 
been,  i.e.  like  some  kind  of  concrete,  unshakeable  law.  Yet,  this  fact  seems  to  have  been  overlooked  by  numerous 
practitioners  in  the  field  who  have,  since  the  inception  of  Dale’s  Cone,  often  referred  to  it  and  used  it  as  if  it  were  a 
road  map. 

Dale  himself  provided  very  few  constraints  to  his  model.  He  clearly  specified  that  it  was  “not  offered  as  a 
perfect  or  mechanically  flawless  picture  to  be  taken  with  absolute  literalness  inks  simplified  form”  (1946,  p.  37). 

He  did  not  provide  any  concrete  propositions  to  guide  practice  either.  For  example,  “Does  the  cone  device  mean  that 
all  experience  must ...  ‘travel  up’  this  routine  from  base  to  pinnacle?  Clearly  and  emphatically  no”  (p.  48).  In  other 
words,  the  Cone’s  intention  was  merely  “to  show  how  sensory  aids  are  classified  in  terms  of  more  or  less 
concreteness  and  abstractness”  (p.  47).  Meanwhile,  Dale  exacerbated  the  fuzziness  of  his  model  by  saying  that  the 
Cone’s  bands  “interlap  (sic)  and  ...  blend  into  one  another”  (p.  52). 

Dale  (1946,  p.  48)  even  quoted  Tyler  (1933)  to  admit  that  (based  on  experiments  in  the  teaching  of 
zoology)  “no  one  series  of  learning  activities  has  proved  equally  effective  with  all  students.”  Similarly,  Dale  refused 
to  make  any  propositions  about  whether  one  learning  activity  was  more  effective  than  the  other,  quoting  Gates 
(1930)  to  say  that  “(t)he  relative  values  of  (different  learning  activities)  are  . . . determined  (not  by  their  degree  of 
abstractness/concreteness  but)  by  past  experience  and  mechanical  advantages”  (p.  50).  Furthermore,  for  all  his 
claims  of  the  “broad  base  that  direct  experience  provides  for  learning  and  communication”  (1969,  p.  109),  Dale 
nevertheless  agreed  with  Dewey  (1916)  that  direct  ejqterience  had  “the  disadvantage  of  being  limited  in  range”  and 
was  “fatally  restricted”  (1946,  p.  51). 

Dale’s  original  presentation  of  the  Cone  was  remarkably  tentative,  and  was  wide  open  to  suggestions  and 
criticism.  “How  would  you  revise  this  device?  Why?”  he  asked  (1946,  p.  52).  He  even  introduced  a possible 
dimension  of  divergence  into  the  model,  asking:  “As  we  become  more  mature,  can  we  concentrate  on  abstract  and 
symbolic  experience,  and  consider  the  getting  of  direct  concrete  experience  as  something  to  be  done  in  childhood?” 
(p.  52).  Dale  did  not  provide  a satisfactory  answer  to  the  latter  question  in  the  subsequent  editions  of  his  book  either, 
merely  stating  that  the  Cone  “stands  for  activities  that  are  available,  in  varying  degrees,  to  learners  in  all  age  groups” 
(1969,  p.  132).  In  Dale’s  own  admission,  his  Cone  device  had  no  answer  to  questions  such  as  “When  ...  is  a student 
ready  for  a generalization?  Should  we  first  (state)  general  principles  or  (rely)  primarily  on  illustrative  examples? 
When  is  ...  rote  learning  permissible?”  (1969,  p.  130). 

However,  these  conceptual  holes  seem  to  have  been  largely  ignored  by  practitioners  (and  their  informants) 
in  their  rush  to  suggest  the  design  of  instruction  on  the  basis  of  propositions  somehow  “implied”  by  Dale’s  model.  In 
fact.  Dale  seems  to  have  disregarded  these  facts  himself,  in  his  1969  book,  when  he  claimed  that  “the  Cone  can  be  a 
helpful  and  practical  guide,”  (p.  1 10)  without  explicitly  detailing  how.  For  instance.  Dale  said  that  in  teaching  one 
does  “not  always  begin  with  direct  experience  at  the  base  of  the  cone,”  but  rather  “with  the  kind  of  experience  that  is 
most  appropriate  to  the  needs  and  abilities  of  a particular  learner  in  a particular  learning  situation”  (1969,  p.  128).  A 
logical  question  from  the  practitioner’s  standpoint  could  then  be,  ‘If  the  Cone  cannot  tell  me  categorically  when, 
where,  and  under  what  conditions  I should  or  should  not  use  direct  experience  (or  any  other  learning  activity)  as  an 
instructional  strategy,  then  of  what  real  use  is  it  as  a practitioner’s  guide?’  Meanwhile,  regarding  whether  one  kind 
of  audio-visual  learning  experience  was  instructionally  more  useful  than  another.  Dale  had  no  concrete  answer, 
advising  merely  that  “varied  types  of  sense  experiences  should  be  provided  in  the  classroom”  (1969,  p.  129).  One 
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could  argue  that  this  rather  shallow  piece  of  advice  was  a mere  restatement  of  something  that  teachers  had  known 
for  a long  time  before  Dale  conjured  up  the  Cone  device,  and  therefore  is  not  particularly  illuminating  as  far  as  the 
practitioner  is  concerned. 

To  give  another  example,  it  is  reasonable  to  expect  that  practitioners  would  like  to  have  concrete  advice 
regarding  the  relative  cognitive  demands  imposed  upon  the  learner  by  each  type  of  audio-visual  learning  exp  erience. 
However,  Dale  said  that  “the  narrowing  upward  shape  of  the  Cone  does  not  imply  an  increasing  difficulty  of 
learning,’’  and  that  “the  basis  of  (his)  classification  is  not  difficulty  but  degree  of  abstraction”  (1969,  p.  1 10).  This 
immediately  raises  the  classic  ‘So  what,  who  cares?’  question,  because  it  can  be  argued  that  merely  classifying 
learning  experiences  on  the  basis  of  degree  of  abstraction  does  is  not  saying  much  to  a practitioner  who  is  looking 
for  a guide  to  help  him/her  make  vital,  pressing  instructional  decisions. 

Examining  Dale’s  Cone  from  Other  Philosophical  Standpoints 

This  discussion  could  possibly  be  enriched  by  examining  the  empirical  usefulness  of  Dale’s  Cone  from  a 
Popperian  perspective.  As  Karl  Popper  saw  it,  if  a theory  does  not  lend  itself  to  falsification,  it  is  more  metaphysical 
than  scientific.  It  could  be  argued  that  Dale’s  Cone  is  indeed  such  a theory.  It  does  not  make  any  “risky”  predictions 
in  the  Popperian  sense,  being  so  imprecise  in  terms  of  its  conditions  and  so  fuzzy  in  terms  of  its  propositions  as  to 
practically  impossible  to  test  or  falsify.  Further,  according  to  Popper,  “every  genuine  scientific  theory  is  prohibitive, 
in  the  sense  that  it  forbids,  by  implication,  particular  events  or  occurrences”  (Thornton,  2001).  However,  Dale’s 
Cone  does  not  forbid  anything  to  happen.  Popper  would  therefore  be  likely  to  frown  upon  the  way  Dale’s  Cone  has 
been  used  by  scholars  and  practitioners,  as  if  it  were  a real  scientific  theory.  It  has  to  be  admitted  that  Dale’s  Cone  of 
Experience  theory  certainly  has  excellent  face  validity.  After  all,  no  one  can  argue  with  the  logic  of  a ladder  of 
abstraction  leading  from  direct  experience  to  visual  symbolism.  Nonetheless,  this  high  level  of  probability  would 
most  likely  not  convince  Popper,  who  believed  that  “the  probability  and  informative  content  of  a theory  vary 
inversely”  (Thornton,  2001),  with  the  most  informative  theories  being  the  ones  that  are  the  most  improbable- 
sounding. 

Meanwhile,  another  argument  against  the  prescriptive  use  of  Dale’s  Cone  is  the  fact  that  there  is  very  little 
by  way  of  an  “empirical  tradition”  associated  with  this  theory.  According  to  Imre  Lakatos,  a theory  would  be 
considered  “empirically  progressive”  if  it  led  “to  the  actual  discovery  of  some  new  fact”  (quoted  in  Court,  1999,  p. 
212).  Reading  Dale’s  explication  of  the  Cone  of  Experience  model  from  a Lakatosian  perspective  will  make  it  clear 
that  it  does  not  have  much  empirical  content  to  offer  (nor  does  it  offer  any  real  new  “facts”)  and  is  therefore  not 
likely  to  be  the  most  effective  practitioner’s  guide.  Lakatos  observed  that  social  scientific  theories  are  often  judged 
by  “assessing  the  number,  faith  and  vocal  energy  of  its  supporters”  (quoted  in  Court,  1999,  p.  215).  According  to 
Court  (1999,  p.  217)  especially  in  our  field  there  is  a risk  of  “assuming  a theory  is  good  because  it  is  popular.”  Court 
describes  instances  of  “bandwagon”  theories,  unsupported  by  sufficient  research,  sweeping  into  town  and  being 
uncritically  embraced.  Clearly  this  has  been  the  case  with  the  widespread  misunderstanding  and  misapplication  of 
Dale’s  Cone  by  practitioners. 

The  main  reason  for  this  state  of  affairs  is  probably  the  notable  lack  of  theoretical  dialogue  in  our  field.  As 
Schwen  (2002,  personal  communication)  says,  “the  discussion  of  theory  in  our  literature  is  flaccid  at  best,”  the  result 
being  the  human  performance  technologists’  claim  that  we  promote  an  outdated  view  of  the  world.  Schwen  has 
suggested  the  use  of  Dubin’s  (1976)  conception  of  the  elements  of  a theoretical  model  as  “intellectual  tools”  by 
which  scholars  could  “deconstruct”  theories.  According  to  Dubin  (1976,  p.  23),  a (complete)  theoretical  model 
involves  Units  (“Variables  ...  whose  interactions  constitute  the  subject-matter  of  attention”);  Laws  of  Interaction 
(“Among  the  units  of  the  model”);  Boundaries  (“Within  which  the  theory  is  expected  to  hold”);  System  States  (“In 
each  of  which  the  units  interact  differently”);  Propositions  (“Conclusions  that  represent  logical  and  true  deductions 
about  the  model  in  operation”);  Empirical  Indicators  (“Each  term  in  each  proposition  whose  test  is  sought  needs  to 
be  converted  into  an  empirical  indicator  of  the  term”);  and  Hypotheses  (Substituting  “the  appropriate  empirical 
indicators  in  the  propositional  statement ...  generate(s)  a testable  hypothesis”). 

It  may  thus  be  a worthwhile  exercise  to  carry  out  a Dubin  -esque  deconstruction  of  Dale’ s explication  of  the 
Cone  of  Experience  into  its  constituent  units,  laws  of  interaction,  boundaries,  system  states,  propositions,  empirical 
indicators,  and  hypotheses.  This  would  help  expose  the  gaps  in  the  prevalent  version  of  Dale’s  Cone,  and  provide  an 
indication  of  what  needs  to  be  done  in  order  to  plug  these  gaps  and  make  the  model  a more  effective  guide  for 
practitioners. 

Deconstructing  Dale’s  Cone  using  Dubin’s  Elements 

Attempting  a deconstruction  of  Dale’s  Cone  using  Dubin’s  elements  of  an  ideal  theoretical  model  is  a very 
illuminative  exercise,  because  it  immediately  exposes  the  extensive  holes  in  Dale’ s model  and  reveals  its  remarkable 
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fuzziness  and  extraordinary  lack  of  informative  content.  A careful  analysis  of  Dale’s  work  divulges  the  following 
units:  Learning  Experiences;  Audio-Visual  Instructional  Materials;  Levels  of  the  Cone;  Degree  of  Experiential 
Concreteness;  and  Methods  for  Teaching  Abstract  Concepts.  However,  the  only  discernible  law  of  Interaction 
among  any  of  the  above  units  of  the  model  seems  to  be  that  ‘there  is  an  inverse  relationship  between  the  Level  of  the 
Cone  and  the  Degree  of  Experiential  Concreteness.’ 

Dale  provided  some  very  broad  boundaries  within  which  he  expected  the  Cone  model  to  hold.  In  “a 
specific  case,”  the  Cone  could  describe  the  range  of  experiences  involved  in  ‘‘a  child’s  eating  of  a kumquat  to  his 
recognition  of  a picture  of  a kumquat  to  his  understanding  of  the  word  ‘kumquat’”  (1969,  p.  1 10).  Concurrently, 
‘‘over  a lifetime  of  learning,”  the  Cone  could  symbolize  the  experiences  involved  in  a ‘‘youngster’ s first  playing  with 
blocks  to  his  understanding  of  a periodic  chart  of  the  chemical  elements  and  from  there  to  his  recognition  of  the 
formula  e=mc2”  (p.  110-111).  Meanwhile,  regarding  the  system  states  in  each  of  which  the  Cone’s  units  interact 
differently,  these  were  variously  described  across  Dale’s  works  as  ‘‘the  needs  and  abilities  of  a particular  learner  in  a 
particular  learning  situation”  (1969,  p.  128),  the  learner’s  ‘‘age,  ability,  and  background”  (p.  130),  and  the  learner’s 
‘‘past  experience  and  mechanical  advantages”  (1946,  p.  50). 

A Dubin-esque  deconstruction  along  thes  e lines  reveals  that  concrete  propositions  are  the  department  in 
which  Dale’s  Cone  is  most  lacking.  The  only  propositions  that  present  themselves  after  a thorough  analysis  of 
Dale’s  works  are  that  ‘‘varied  types  of  sense  experiences  should  be  provided  in  the  classroom”  (1969,  p.  129),  and 
that  the  Cone  ‘‘stands  for  activities  that  are  available,  in  varying  degrees,  to  learners  in  all  age  groups”  (1969,  p. 

132).  As  is  immediately  apparent,  these  propositions  are  so  weak,  so  generic,  so  abstract,  that  any  attempt  to  find 
any  empirical  indicators  for  their  constituent  terms  (and  consequently  to  restate  them  as  testable  hypotheses  is  an 
exercise  in  futility.  Eor  Dale’s  Cone  to  be  a meaningful  guide  for  practitioners,  it  needs  to  be  refined  so  that  it  has 
concrete  propositions  and  empirically  testable  hypotheses  to  offer. 

Final  Remarks 

This  paper  attempts  to  bring  to  light  the  fundamental  issues  concerning  one  of  the  most  influential  bodies  of 
theory  in  our  field  during  the  20th  century,  seeking  to  describe  how  the  related  ideas  of  Dale  and  other  theoreticians 
have  emerged  without  evidence  of  much  dialogue  having  taken  place  between  them.  As  a result,  individual  theories 
such  as  Dale’s  Cone  display  significant  fuzziness  and  lacunae,  which  reduce  their  value  as  guides  of  practice. 
Meanwhile,  those  purporting  to  inform  practitioners  have  often  done  the  latter  a disservice  by  blurring  the 
boundaries  of  these  theories  and/or  drawing  inadvisable  inferences  from  them  . . . although  this  may  well  be  due  to 
the  fact  that  these  theories  themselves  are  (in  a Popperian  sense)  unscientific. 

However,  it  must  be  emphasized  at  this  point  that  the  object  of  this  paper  is  not  to  discredit  Dale  and 
devalue  his  Cone  model,  but  rather  to  provide  an  example  of  a state  of  affairs  that  is  widely  prevalent  in  our  field 
today  ...  a state  of  affairs  in  which  theoretical  fuzziness  and  misconception  abounds  due  to  the  lack  of  sufficient 
dialogue  between  scholars.  Consequently,  the  resulting  nomological  network  is  one  that  is  full  of  holes  and  missing 
links.  Meanwhile,  the  waters  are  muddied  further  by  the  fact  that  practitioners  rarely  have  first-hand  access  to 
original  works,  but  have  to  rely  on  second-hand  sources  ...  indirect  ‘interpretations  of  interpretations’  that  are  very 
often  far  from  accurate  reflections  of  what  the  original  author  had  to  say. 
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Introduction 

The  importance  of  technology  continues  to  grow  in  American  society.  From  e-mail  and  cell  phones  to  laser 
surgery,  technology  seems  to  touch  all  phases  of  life  today.  As  in  business  and  medicine,  the  uses  of  technology  in 
education  have  grown  dramatically  in  the  last  decade.  Huge  numbers  of  schools  have  been  wired  for 
telecommunication  access  in  the  last  three  years  alone.  As  of  fall  1998,  90%  of  schools  had  Internet  access  and  39% 
of  teachers  had  access  to  the  Internet  in  their  own  classrooms  (Becker,  1998).  The  numbers  of  computers  in  U.S. 
schools  also  continues  to  increase  with  more  than  six  million  computers  in  the  nation's  elementary  and  secondary 
schools  today  (Fulton,  1999).  Many  states,  too,  have  now  established  standards  for  the  use  of  technology  by  their 
students.  In  one  major  national  report  on  school  technology  and  readiness,  it  was  pointed  out  that  all  students  must 
graduate  with  the  technology  skills  needed  in  today's  world  and  tomorrow's  workplace  and  all  educators  must  be 
equipped  to  use  technology  as  a tool  to  achieve  high  academic  standards  (CEO  Forum,  1999).  Furthermore,  many 
states  have  now  adopted  specific  technology  requirements  for  students  preparing  to  teach  and  stress  the  use  of 
technology  in  their  schools  based  on  the  new  standards  of  the  International  Society  for  Technology  in  Education 
(ISTE).  Accordingly,  most  educators  today  believe  that  public  schools  must  ensure  the  effective  use  of  technology 
in  the  classroom  in  order  to  transform  our  children  for  2T‘  century  learning  environments. 

In  spite  of  the  tremendous  growth  of  technology  in  our  nation's  schools  and  the  belief  by  a majority  of 
educators  that  all  students  must  have  access  to  technology  to  be  truly  successful  in  today's  world,  there  is  evidence 
that  many  teachers  still  do  not  use  technology  at  all  in  their  teaching  (Education  Week,  1999).  In  addition,  there  are 
many  questions  regarding  the  values  and  uses  of  technology  in  schools.  Are  teachers  adequately  prepared  to  use 
technology  in  their  teaching?  Are  schools  providing  appropriate  types  of  technology  for  teachers  and  students?  Do 
teachers  really  value  technology  in  their  teaching?  What  types  of  technology  do  teachers  really  use  in  their 
classrooms?  There  also  are  major  questions  dealing  with  the  role  of  technology  in  schools.  Eor  example,  what  are 
the  goals  of  using  technology  in  elementary,  middle  and  high  schools?  How  can  technology  improve  student 
learning?  Middle  schools,  in  particular,  can  play  an  important  role  in  helping  students  become  skilled  in  the  use  of 
technology.  Not  only  can  technology  help  middle  graders  learn  more  effectively,  its  use  can  also  enable  students  to 
improve  their  skills. 

To  answer  many  of  the  questions  surrounding  technology  in  schools,  a study  was  conducted  in  1,000 
middle  schools  in  ten  states  (Michigan,  Pennsylvania,  Arizona,  Oregon,  Maine,  Ohio,  Georgia,  North  Carolina,  New 
York,  and  Wisconsin).  This  study  explored  the  types  of  technology  used  by  middle  school  teachers,  how  they  value 
the  technology,  and  the  facilities  and  equipment  provided  by  their  schools.  One  thousand  questionnaires  were  sent 
to  these  teachers  and  239  were  returned. 

The  questionnaire  consisted  of  two  parts.  In  Part  One  the  teachers  marked  how  often  they  used  particular 
types  of  technology  (never,  rarely,  sometimes  and  often)  and  how  valuable  these  types  of  technology  were  (very 
valuable,  valuable,  no  opinion,  little  value  and  no  value).  In  Part  Two  the  subjects  indicated  the  availability  of 
different  types  of  technology  in  their  schools  and  the  training  provided  for  them  and  their  students  by  their  schools. 
The  results  were  analyzed  using  percentages  of  use  and  of  value.  The  majority  of  those  responding  were  female  and 
most  had  one  to  ten  years  of  experience.  Moreover,  most  of  the  respondents  were  mathematics  and  science  teachers 
followed  by  those  teaching  social  studies  and  English. 

Results 

The  results  of  this  study  indicate  several  important  findings  concerning  technology  in  middle  schools. 

Table  A shows  that  the  most  often  used  types  of  technology  were:  word  processing  for  teaching  (81.7%),  computers 
in  preparing  lessons  (70.2%),  and  the  overhead  projector  (56. 1%).  The  next  most  often  used  forms  of  technology 
were:  the  Internet  for  teaching  (50.9%),  using  e-mail  to  discuss  teaching  with  other  educators  (44%),  computer 
assisted  instruction  (39.9%)  and  computer  graphics  for  teaching  (39.7).  Table  A also  reveals  that  there  were  other 
types  of  technology  used  less  often  by  the  teachers:  VCR's  (3 1.7%),  Microsoft  Power  Point  and  other  types  of 
computer  programs  (33.5%),  spreadsheets  for  assisting  teaching  (29.5%),  CD-ROM  disks  (27.4),  television  (26.6%), 
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web  pages  (22.4%),  video  projector  (LCD  projectors)  (21.5%),  database  to  assist  teaching  (18.3%),  digital  camera 
(23.1%),  and  films  (15.2%).  Additional  evidence  found  in  Table  A indicates  that  the  least  often  used  types  of 
technology  were:  the  tape  recorder  (8.5%),  digital  camcorder  (6.8%),  laser  disc  player  (5.8%),  radio  (4.9%),  film 
strips  (4.1%),  and  slide  projector  (2.7%). 


Table  A:  Technology  Used  by  Teachers 


Often 

Sometimes 

Rarely 

Never 

Use  of  word  processing  for  teaching 

81.7% 

13.8% 

3.1% 

1.3% 

Use  of  computer  to  prepare  lessons 

70.2% 

23.6% 

4.9% 

1.3% 

Use  of  overhead  projector  in  teaching 

56.1% 

21.1% 

11.2% 

11.7% 

Use  of  Internet  for  teaching 

50.9% 

37.4% 

7.7% 

4.1% 

Using  e-mail  to  discuss  teaching  with  other  educators 

44.0% 

32.4% 

14.7% 

8.4% 

Use  of  computer  assisted  instruction 

39.9% 

39.9% 

14.8% 

5.4% 

Use  of  computer  graphics  for  teaching 

39.7% 

38.4% 

13.8% 

8.0% 

Use  of  VCR  in  Teaching 

31.7% 

41.1% 

19.6% 

7.6% 

Use  of  Microsoft  Power  Point  and  other  computer 
programs  for  teaching 

33.5% 

29.0% 

13.8% 

23.7% 

Use  of  spreadsheets  for  assisting  teaching 

29.5% 

34.8% 

22.3% 

13.4% 

Use  of  CD-ROM  in  teaching 

27.4% 

38.1% 

21.1% 

13.5% 

Use  of  television  in  teaching 

26.6% 

41.0% 

21.6% 

10.8% 

Create  web  pages  for  teaching 

22.4% 

22.0% 

10.8% 

44.8% 

Use  of  video  projector  in  teaching 

21.5% 

26.0% 

19.3% 

33.2% 

Use  of  database  to  assist  teaching 

18.3% 

29.9% 

23.7% 

28.1% 

Use  of  digital  camera  in  teaching 

23.7% 

31.3% 

15.6% 

29.5% 

Use  of  films  in  teaching 

15.2% 

30.8% 

24.1% 

29.9% 

Use  of  tape  recorder  in  teaching 

8.5% 

27.7% 

25.4% 

38.4% 

Use  of  digital  camcorder  in  teaching 

6.8% 

17.1% 

21.2% 

55.0% 

Use  of  laser  disc  player  in  teaching 

5.8% 

9.4% 

13.8% 

71.0% 

Use  of  radio  in  teaching 

4.9% 

16.1% 

23.2% 

55.4% 

Use  of  film  strips  in  teaching 

4.1% 

9.0% 

18.9% 

68.0% 

Use  of  slide  projector  in  teaching 

2.7% 

9.5% 

21.2% 

66.2% 

Use  of  record  player  in  teaching 

2.2% 

4.0% 

13.9% 

79.8% 

Results  reported  in  Table  B show  the  different  types  of  technology  required  or  encouraged  by  teachers  for  their 
students:  encourage  student  use  of  Internet  for  work  (55.2%),  require  computers  for  learning  or  assignments 
(49. 1%),  require  student  to  use  word  processing  for  assignments  (38.7%),  require  use  of  Internet  for  work  (34.4%), 
encourage  power  point  use  by  students  (29.9%)  encourage  students  to  use  computer  graphics  (29.6%),  encourage 
use  of  different  types  of  media  in  work  (21.4%),  encourage  students  to  use  CD-ROM  for  work  (20.6%).  Further 
results  in  Table  B show  that  other  lesser  used  types  of  technology  by  students  were:  encourage  e-mail  use  by 
students  (13.4%),  spreadsheets  (9.9%),  database  (9.6%),  VCR  (7.3%)  and  digital  camcorder  (4.5%),  and  laser  disc 
players  (3.6%). 


Table  B:  Technology  Encouraged  or  Required  for  Students 


Often 

Sometimes 

Rarely 

Never 

Encourage  student  use  of  Internet  for  work 

55.2% 

33.6% 

8.1% 

3.1% 

Require  computers  for  learning  or  assignments 

49.1% 

36.2% 

8.0% 

6.7% 

Require  students  to  use  word  processing 

38.7% 

37.8% 

13.3% 

10.2% 

Require  use  of  Internet  for  work 

34.4% 

37.9% 

10.3% 

17.4% 

Encourage  power  point  use  by  students 

29.9% 

29.9% 

13.8% 

26.3% 

Encourage  students  to  use  computer  graphics 

29.6% 

42.6% 

13.5% 

14.3% 

Encourage  use  of  different  types  of  media  in  work 

21.4% 

33.5% 

21.9% 

21.9% 

Encourage  student  to  use  CD-ROM  for  work 

20.6% 

34.1% 

17.9% 

27.4% 

E-Mail  use  by  students 

13.4% 

24.1% 

28.1% 

34.4% 

Student  use  of  spreadsheets 

9.9% 

26.5% 

33.2% 

30.5% 
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Student  use  of  database 

9.6% 

18.3% 

29.8% 

42.2% 

Student  use  of  VCR 

7.3% 

26.5% 

32.4% 

33.8% 

Encourage  student  use  of  digital  camcorder 

4.5% 

18.6% 

21.4% 

55.5% 

Encourage  student  use  of  laser  disc  players 

3.6% 

5.8% 

9.9% 

80.7% 

Findings  in  Table  C indicate  that  schools  are  providing  many  types  of  technology  for  their  teachers.  VCR's  are 
provided  for  a majority  of  teachers  (92.0%)  and  Internet  access  is  provided  in  most  classrooms  (90.2%).  CD-ROM 
equipment  is  available  in  nearly  eighty  percent  of  the  classrooms  (82.3%)  and  computers  are  provided  for  students 
in  more  than  half  of  the  classrooms  (72.0%),  along  with  digital  cameras  (68.4%)  and  e-mail  facilities  which  are 
available  in  many  classrooms  (80.4%).  Video  (LCD)  projectors  are  available  in  several  schools  (54.5%),  while 
training  for  teachers  (90.2%)  and  students  (92.9%)  in  the  use  of  technology  is  provided  by  most  schools. 


Table  C:  Technology  Provided  for  Teachers  and  Students 


Yes 

No 

VCR's  provided  in  classroom 

92.0% 

7.6% 

Internet  access  in  classroom 

90.2% 

9.3% 

CD-ROM  equipment  provided  in  classroom 

82.3% 

17.7% 

Computers  provided  in  classroom 

72.0% 

28.0% 

Digital  cameras  provided  in  classroom 

68.4% 

31.6% 

E-Mail  in  classroom 

80.4% 

19.2% 

Video  (LCD)  projectors  provided  in  classroom 

54.5% 

45.5% 

Training  for  teachers  to  use  computers 

92.9% 

7.1% 

Training  for  students 

90.2% 

9.4% 

Conclusions 

Evidence  from  this  study  indicates  several  important  findings  concerning  technology  in  middle  schools. 
First,  middle  school  teachers  are  using  technology  in  their  classrooms.  Computers  seem  to  be  the  most  popular 
choice  with  over  70%  of  the  teachers  reporting  that  they  often  use  them  in  their  teaching.  This  finding  is  important 
because  it  supports  the  views  of  most  contemporary  educators  and  the  public  who  want  more  technology  in  schools. 
Second,  middle  school  teachers  are  using  many  types  of  technology  in  their  classroom.  Beside  computers,  teachers 
are  using  e-mail,  VCR's,  Microsoft  Power  Point,  CD-ROM's,  digital  cameras,  and  other  contemporary  form  of 
technology.  The  result  is  significant  because  it  supports  the  demands  for  greater  use  of  technology  in  education  by 
educators,  parents,  and  the  public. 

Third,  middle  school  teachers  are  encouraging  and  requiring  that  their  student  use  technology  in  their  work. 
This  result,  too,  is  important  because  it  reinforces  the  belief  that  students  need  technology  to  aid  their  learning  and  to 
develop  technology  skills  needed  in  later  life. 

Fourth,  middle  school  teachers  and  students  are  receiving  technology  resources  and  training  from  their 
schools.  This  finding  is  crucial  because  without  these  essentials  teachers  and  students  could  not  successfully  use 
technology  in  their  work. 

Although  the  study  indicates  that  middle  school  teachers  are  using  technology  in  their  teaching, 
encouraging  their  students  to  use  technology,  and  are  being  provided  essential  resources,  equipment,  and  training.  It 
is  also  clear  that  these  teachers  need  to  continue  and  increase  their  uses  of  technology  in  their  classrooms.  Students, 
too,  must  continue  to  improve  their  skills  in  using  technology  by  using  it  for  their  learning.  Schools,  also,  must 
continue  to  provide  different  types  of  technology,  resources,  facilities  and  training  for  both  teachers  and  students. 

As  we  enter  the  2L*  century,  it  is  hoped  that  all  middle  school  teachers  will  make  technology  an  essential  part  of 
their  teaching  and  that  they  will  strive  to  help  their  students  become  "technologically  literate." 
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Abstract 

Currently  there  is  a growing  interest  in  Web-based  multimedia  learning  environments,  particularly  those 
making  use  of  asynchronous  streaming  video.  This  interest  motivates  renewed  attention  to  properties  of  video  for 
educational  purposes.  A typical  property  of  video  is  its  emotion-evoking  potential.  Research  by  Isen,  Daubman,  and 
Nowicki  (1987),  Kaufmann  and  Vosburg  ( 1997)  and  by  Vosburg  (1998)  on  video-evoked  positive  or  negative  mood 
states  inspired  a research  project  on  the  didactical  functionality  of  emotion-evoking  video  materials  in  relationship 
to  (educational)  problem  solving  tasks  within  a Web-based  environment.  The  results  show  that  the  video  materials 
that  were  used  in  the  experiment  induced  the  expected  positive  or  negative  mood.  Differential  effects  of  positive  or 
negative  mood  for  problem  solving  tasks,  however,  were  not  observed.  This  outcome  is  discussed  in  the  context  of 
the  findings  of  the  above-mentioned  authors. 

Introduction 

Vosburg  (1998)  reviewed  research  studies  by  Isen,  Daubman,  and  Nowicki  (1987)  and  Kaufmann  and 
Vosburg  (1997,  Study  2)  that  show  that  video  segments  that  are  used  to  evoke  affective  responses  can  influence  the 
performance  on  a problem  solving  task  that  follows  the  video  segment.  These  studies  pertain  to  positive  and 
negative  mood  states  that  may  be  related  to  the  way  a creative  problem-solving  task  is  carried  out.  In  all  cases,  the 
content  of  the  video  segments  was  not  related  to  the  task.  The  results  of  these  studies  seem  in  a first  instance  to  lead 
to  contradictory  conclusions.  Isen,  Daubman,  and  Nowicki  found  that  positive  mood  had  a facilitative  effect  on 
creative  problem  solving,  while  Kaufmann  and  Vosburg  found  that  positive  mood  could  significantly  inhibit  creative 
problem  solving.  Vosburg  (1998)  proposes  a model  that  may  help  to  get  a better  understanding  of  the  underlying 
phenomena  and  thus  provide  an  explanation  for  the  earlier  results.  It  is  a satisficing/optimizing  model  that  describes 
differential  effects  of  mood  states  (positive  versus  negative)  on  problem  solving  tasks  as  a result  of  given  task 
requirements  or  conditions.  Kaufmann  and  Vosburg  (1997)  suggest  that  optimizing  and  satisficing  strategies  may  be 
important  moderators  of  this  mood  influence.  The  purpose  of  the  current  study  is  to  explore  whether  the 
satisficing/optimizing  model  could  be  verified  as  a first  step  towards  the  development  of  didactical  functions  of 
emotion-evoking  video  materials.  The  two  types  of  task  requirements  or  conditions  are  described  as  follows 
(adapted  from  Vosburg,  1998,  pl66-167): 

Under  satisficing  conditions  the  requirements  for  the  solution  of  the  problem-solving  task  are  open.  When  a 
participant  is  satisfied  with  his  or  her  answer  according  to  self-selected  subjective  criteria  the  task  is  finished.  An 
example  of  a task  with  a typical  satisficing  requirement  is  a divergent  thinking  problem.  In  this  kind  of  task 
participants  are  asked  to  generate  “as  much  solutions  as  possible  for  a presented  problem  that  they  can  think  of  and 
not  to  think  of  the  quality  of  their  responses". 

Under  optimizing  conditions  participants  are  asked  to  present  the  best  solution  for  a given  problem  that  fits 
the  strict  solution  requirements.  Insight  problems  exemplify  problem  solving  under  optimizing  conditions,  because 
strict  solution  requirements  exist  and  there  is  (often)  only  one  acceptable  solution.  An  example  of  such  a task  is 
Duncker’s  candle  task  (Duncker,  1945).  On  this  task,  subjects  get  a candle,  a box  with  tacks,  and  a book  of  matches. 
Their  task  is  to  mount  the  candle  against  a wall  so  that  it  can  burn  without  dripping.  How  to  do  that  with  the  help  of 
the  given  items,  is  the  problem  to  solve. 

The  satisficing/optimizing  model  proposes  that  positive  mood  tends  to  lower  and  negative  mood  to  raise 
subjective  criteria  for  acceptable  solutions.  Positive  moods  are  expected  to  facilitate  and  negative  moods  to  inhibit 
task  performance  under  satisficing  conditions  (for  instance  a divergent  thinking  task).  Negative  moods  are  expected 
to  facilitate  and  positive  moods  to  inhibit  task  performance  under  optimizing  conditions  (for  instance  an  insight 
task). 

In  the  present  study  both  video,  as  mood  induction  technique,  and  two  creative  problem-solving  tasks  (a 
divergent  and  an  insight  task)  are  embedded  within  a Web-based  environment  as  opposed  to  more  traditional 
settings  in  the  comparable  studies.  If  the  model  can  be  verified,  this  mood-facilitating  approach  to  problem  solving 
could  als  o be  relevant  for  realistic  problem  solving  tasks  in  Web-based  E-learning  environments. 
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(The  preliminary  question  whether  the  affect-evoking  potential  of  video  presented  in  a Web-environment  is 
comparable  with  video  presented  in  a more  traditional  context,  has  been  answered  earlier  by  Verleur  and  Verhagen, 
2001,  who  conducted  an  experiment  that  demonstrated  that  Web-based  affective  video  can  be  an  effective  mood- 
induction  technique.). 

The  purpose  of  the  current  study  is  operationalised  in  the  following  research  question: 

When  both  the  mood-evoking  video  materials  and  the  problem  solving  tasks  are  embedded  in  a Web-based 
environment,  can  the  "satisficing-optimizing  model”  (Vosburg,  1998)  be  verified? 

This  question  is  answered  by  testing  the  following  hypotheses  (see  also  Table  1)  in  a Web-based 
environment: 

1)  Compared  to  a negative  video  clip,  a positive  video  clip  will  result  in: 

A better  performance  on  the  divergent  thinking  task  that  follows  the  clip; 
a worse  performance  on  the  insight  task  that  follows  the  clip. 

2)  Compared  to  a positive  video  clip,  a negative  video  clip  will  result  in: 

A better  performance  on  the  insight  task  that  follows  the  clip; 

a worse  performance  on  the  divergent  thinking  task  that  follows  the  clip. 

Table  1.  Overview  of  the  expected  influence  of  affect-inducing  video 


clips  on  task  performance  in  a Web-based  environment 


Insight  task 

Divergent  -thinking 
task 

Positive  video 

- 

-1- 

clip 

Negative  video  clip 

+ 

- 

Note: 

-I-  = Facilitates  performance 
- = Inhibits  performance 


Method 

To  study  the  hypotheses,  an  experiment  was  carried  out  that  is  specified  below. 

Participants 

101  first-year  university  students  in  communication  studies  (70  women  and  31  men,  mean  age  = 18.67 
years)  participated  in  the  experiment  that  was  part  of  a course  that  introduces  the  field  of  media  communication  and 
media  research.  The  results  of  the  experiment  and  the  experience  of  the  students  as  subjects  for  the  experiment  were 
discussed  in  class  a month  after  the  experiment. 

Materials 

For  the  selection  of  affect-evoking  video  materials,  video  segments  of  about  two  or  three  minutes  were 
used.  In  the  experimental  setting  the  positive  video  segment  was  taken  from  the  comedy  movie  ‘When  Harry  met 
Sally’  (2’45”);  the  negative  video  segment  was  a news-item  about  ‘Hunger  in  Ethiopia’  (2’04”).  To  establish  a 
comparable  starting  position  for  all  subjects  a 'neutral'  video  segment  was  added  as  a first  video  clip  for  all 
participants.  This  segment  was  a part  of  a documentary  about  birds  (‘1’53”).  All  three  clips  were  also  used  in  the 
experiment  reported  by  Verleur  and  Verhagen  (2001). 

Two  types  of  tasks  are  used  to  test  the  satisficing-optimizing  model:  A divergent  thinking  task  as  a typical 
example  of  a task  with  a strict  satisficing  requirement  and  an  insight  task  as  a typical  example  of  a task  with  a strict 
optimizing  requirement.  Task  selection  started  from  a replication  perspective  with  considering  the  creative  problem 
solving  tasks  from  the  studies  of  Isen,  Daubman,  and  Nowicki  (1987),  Kaufmann  and  Vo  sburg  (1997)  and  Vosburg 
(1998).  Although  the  Duncker  Candle  task  that  was  used  in  the  study  of  Isen,  Daubman,  and  Nowicki  (1987)  has 
attractive  properties  to  be  used  for  the  insight  task,  it  was  not  selected  because  the  original  task  with  realia  cannot 
easily  be  embedded  in  a Web-environment.  The  chosen  insight  task  is  the  "Two  String  Problem"  from  Maier  (1970). 
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This  task  was  the  best  performing  insight  task  in  the  study  of  Kaufmann  and  Vosburg  (1997).  The  chosen  divergent 
thinking  task  is  the  "Real-life  divergent-thinking  task"  that  was  described  in  the  study  of  Vosburg  (1998).  This  task 
is  referred  to  as  the  "Class  problem"  (how  to  handle  being  distracted  during  a lecture  class)  adapted  from  Mraz  and 
Runco  (1994). 

In  the  Two  String  problem  the  task  is  to  tie  together  two  strings  hanging  down  from  the  ceiling.  The  two 
strings  are  too  far  apart  to  be  reached  by  hands  alone.  The  participants  have  to  solve  this  problem  by  using  tools  that 
they  select  from  a set  of  items.  The  available  items  are  a pair  of  pliers,  a screwdriver  and  a box  of  tacks.  The  correct 
solution  is  to  tie  one  or  each  string  to  one  of  the  tools  (i.e.  the  pair  of  pliers  or  the  screwdriver),  push  one  or  both  into 
a pendulum  movement  and  to  grab  the  string(s)  when  they  are  close  enough.  Following  Martinsen's  version  of  the 
task  (1993)  two  points  were  given  for  this  solution;  one  point  for  a good  try  (for  instance  when  the  tacks  were  used 
to  stitch  one  or  both  strings  to  the  wall  to  get  them  closer  to  each  other);  and  no  points  for  a wrong  or  no  solution. 
Hence  the  score  for  these  tasks  could  be  2,  1,  or  0.  Conform  the  study  of  Kaufmann  and  Vosburg  (1997)  the  task 
was  presented  in  writing  with  an  illustration  of  the  situation  and  the  tools  available. 

The  Class  problem  is  a verbal  description  of  a realistic  problem  situated  in  a classroom  and  is  written  from 
the  perspective  of  the  problem  solver.  Following  Vosburg 's  version  of  the  task  (1998)  the  description  presented  was 
as  follows:  "Rolf,  a friend  of  yours  sits  next  to  you  in  the  classroom.  Rolf  likes  to  talk  to  you  and  often  interrupts 
you  when  you  are  taking  notes.  Sometimes  he  distracts  you  so  that  you  are  missing  important  parts  of  the  lecture." 
The  accompanying  instruction  asked  the  subjects  to  think  of  different  solutions  to  solve  this  problem:  "What  are  you 
going  to  do?  How  are  you  going  to  solve  this  problem?".  In  a pre-test  with  a task  comparable  to  the  Class  Problem  it 
appeared,  however,  that  subjects  tended  to  generate  a step-wise  procedure  for  one  solution  in  stead  of  generating 
different  solutions.  This  made  it  necessary  to  reformulate  the  instruction  into  "Think  of  different  solutions  to  solve 
this  problem";  to  make  sure  that  students  would  generate  'different  solutions'.  The  instruction  specified  what  was 
mend  by  the  word  'different'. 

Design  and  procedure  An  experiment  was  designed  with  two  conditions:  A 'Positive  Affect'  and  a 'Negative 
Affect'  condition.  In  each  condition  the  participants  encountered  both  the  insight  and  the  divergent  thinking  task.  To 
balance  for  order  effects  the  task  order  was  randomized  within  each  condition.  Table  2 presents  an  overview  of  the 
experimental  conditions  and  the  resulting  four  versions  of  the  experimental  arrangement  that  were  embedded  in  the 
Web-environment.  To  improve  the  precision  for  analyzing  affective  state  effects,  the  study  was  balanced  by  keeping 
the  number  of  subjects  in  each  of  the  two  conditions  equal.  Furthermore  the  subjects  were  blocked  by  gender. 


Table  2.  Overview  of  the  experimental  conditions 


Condition 

Task  order 

(two  versions  for  each  condition) 

Positive  Affect 

Insight  task  - Divergent  thinking  task 

Divergent  thinking  task  - Insight  task 

Negative  Affect 

Insight  task  - Divergent  thinking  task 

Divergent  thinking  task  - Insight  task 

Note:  Insight  task  = Two  String  Problem;  Divergent  thinking  task  = Class  problem 

After  viewing  the  'neutral'  video  clip  a questionnaire  was  presented  to  pre-test  the  initial  affective  state  of 
the  participant.  Then  an  example  problem-solving  task  was  presented  to  familiarize  the  participant  with  this  type  of 
task.  (The  data  of  the  example  problem  are  no  part  of  the  analyses.)  Next  the  experimental  part  of  the  session  started 
by  showing  the  positive  or  negative  affective  video  clip  followed  by  the  same  questionnaire  that  was  used  after  the 
neutral  video  clip  to  measure  the  video-induced  affective  state.  This  was  followed  by  both  problem  tasks  that  were 
presented  in  random  order:  half  of  the  participants  first  encountered  the  insight  task  followed  by  the  divergent 
thinking  task,  for  the  other  half  this  was  the  other  way  around.  For  each  task  a maximum  of  5 minutes  was  available 
to  work  on  the  solution. 
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The  experiment  was  conducted  parallel  in  two  computer  rooms  during  scheduled  computer  lab  hours  with 
all  four  versions  of  the  Web-environment  in  both  rooms  present.  Each  room  contained  20  individual  workplaces. 

The  number  of  participants  required  three  rounds  to  lead  all  participants  through  the  experiment.  Each  round  took 
place  in  a time  slot  of  one  hour  and  15  minutes.  All  participants  were  able  to  complete  their  tasks  within  that  period 
without  further  time  pressure  than  the  timed  answers  to  the  problems.  All  rounds  were  administered  on  the  afternoon 
of  the  same  day. 

Each  individual  workplace  consisted  of  a computer  configuration  with  a 17-inch  monitor  and  a headset. 
Separation  screens  prevented  distractions  by  eye  contact  or  viewing  other  monitors.  The  experimental  Web- 
environment  was  available  as  a click  able  icon  on  the  desktop.  The  experimental  procedure,  instruments  and 
introductions  to  the  steps  in  the  experiment  were  all  embedded  in  the  Web-environment. 

Randomizing  in  advance  each  order  of  four  versions  predetermined  the  order  of  the  Web-version 
assignment.  In  order  to  spread  the  male  participants  equally  over  the  versions,  in  each  round  the  male  group  had  to 
enter  the  room  first  and  was  then  randomly  distributed  over  all  versions.  After  a brief  introduction  to  the  experiment 
the  participants  were  asked  to  click  on  the  icon  on  the  desktop  that  would  launch  the  experimental  Web- 
environment.  The  experimental  procedure,  instruments  and  introductions  to  the  steps  in  the  experiment  were  all 
shown,  explained,  and  practiced  in  an  introductory  part  of  the  Web-environment  before  the  experimental  part  of  the 
study  started. 

Scoring 

Instruments.  To  measure  the  affective  responses  the  paper-and-pencil  version  of  the  Self-Assessment  Manikin 
(SAM)  devised  by  Peter  Lang  (Bradley  & Lang,  1995)  was  adapted  into  a Web-version.  The  original  SAM  measures 
three  emotional  dimensions:  Valance  (ranging  from  pleasant  to  unpleasant),  arousal  (ranging  from  calm  to  excited) 
and  Dominance  (ranging  from  controlled  to  in -control)  with  a 9-points  rating  scale  for  each  dimension.  The  Valence 
dimension  of  the  SAM-scale  was  used  in  the  present  study  as  a measure  for  the  'Affective  State'.  Eigure  1 shows  the 
Web-version  of  this  scale.  In  addition  a small  number  of  bipolar  items  on  a semantic  differential  (also  using  a 9- 
point  scale)  were  added  to  assess  cognitive  and  mood  responses  to  the  video  materials.  These  questions  were  not 
part  of  the  experiment  but  were  used  to  mask  the  purpose  of  the  study 


EIGURE  I.  Web-version  for  the  Valence  dimension  of  the  Self-Assessment  Manikin  (SAM) 

Results 

Homogeneity  and  manipulation  check  for  affective  state 


Initial  affective  state.  During  the  practice  part  of  the  session  the  initial  affective  state  was  measured  just  after  the 
presentation  of  the  'neutral'  video  clip  on  the  9-points  SAM  scale  (1=  unpleasant,  9=  pleasant).  Overall  the  neutral 
clip  resulted  in  a 'light  positive'  affect  state  (V=101,  M=6.41,  SD=1.23).  When  the  two  experimental  conditions  were 
compared,  a t test  showed  no  significant  differences  between  the  positive  («=50,  M=6.46,  SD=)  and  the  negative 
(«=51,  M=6.35,  SD=)  condition  (t[99]=-0.437,  p=.663,  two-tailed).  The  two  conditions  can  be  considered 
homogeneous  on  initial  affective  state. 


Affective  state  changes.  The  affective  state  after  viewing  the  first  'neutral'  clip  was  compared  with  the  affective  state 
after  viewing  the  second  'affective'  clip  (being  positive  or  negative  depending  on  the  condition).  In  the  negative 
condition  the  affective  state  changed  into  a more  negative  state  (AAf=3.06,  SD=\.61-,  AM=Mneutrai  - A^negative)-  A 
paired-T-test  showed  that  this  mean  change  was  significant  (?[50]=I3.5I,  p<0.001,  two-tailed).  In  the  positive 
condition  the  affective  state  changed  into  a more  positive  state  (AM=- 1.58,  50=1.39;  AM=  Mneutrai  - A^positive)- 
Although  the  mean  change  was  less  than  in  the  negative  condition  a paired-T -test  showed  that  this  change  was  also 


436 


significant  (t[49]=-8.06,  /?<0.001,  two-tailed).  The  'light  positive'  nature  of  the  neutral  clip  might  have  been  of 
influence  on  the  magnitude  of  these  changes.  The  manipulation  check  data  confirmed  the  expected  positive  and 
negative  affective  states  in  the  two  conditions  as  result  of  the  affective  video  inductions. 

Resulting  mean  affective  state  scores  for  the  two  conditions.  The  resulting  mean  affective  state  score  was  in  the 
negative  condition  clearly  lower  (M=3.29,  SD=1.43)  than  in  the  positive  condition  (lff=8.04,  SD=\.\2).  A t test 
showed  that  the  difference  between  the  two  conditions  was  significant  (f[99]=4.75,  /?<0.001,  two-tailed). 

Two  String  Problem  (Insight  Task) 

Only  20  (19.8%)  of  all  subjects  found  the  best  solution  for  the  problem.  In  the  negative  condition  12  of  the 
51  subjects  found  the  optimal  solution  to  the  problem,  31  subjects  had  a 'good  try'  score  and  eight  subjects  did  not 
solve  the  problem.  In  the  positive  condition  8 of  the  50  subjects  solved  the  problem,  27  subjects  had  a 'good  try' 
score  and  15  subjects  did  not  solve  the  problem  at  all.  A Mann-Whitney  test  indicated  that  the  differences  between 
the  two  conditions  were  not  significant  (Z=-1.68,p=0.92,  two-tailed).  Solutions  that  were  categorized  under  'good 
try'  (f=5di)  were:  'extending  string  or  arm  with  tools'  (f=25),  'stitching  strings  to  the  wall  or  ceiling'  (f=23),  'pushing 
strings  into  a pendulum  movement  without  using  (sufficient)  weight'  (J=10).  Table  3 summarizes  the  results. 


Table  3.  Frequencies  per  solution  type  for  the  Two  String  Problem 


Condition 

No/wrong 

solution 

Good  try 

Best  solution 

Negative  Affect  (n=51) 

8 

31 

12 

Positive  Affect  (n=50) 

15 

27 

8 

Total  (N=101) 

23 

58 

20 

Class  Problem  (Divergent  Thinking  Task) 

Based  on  the  solutions  given  by  the  subjects  a category  list  was  developed  to  make  a clear  distinction 
between  the  different  solutions/answers.  The  data  showed  that  some  subjects  combined  two  solutions  in  one 
solution,  where  others  presented  them  as  two  separate  solutions.  For  example  two  solutions  might  have  been  'telling 
Rolf  to  stop  talking'  and  'agree  to  meet  Rolf  during  the  break'.  Some  subjects  combined  these  two  solutions  into  one 
sentence.  In  both  cases  two  solutions  were  counted.  (Although  this  procedure  was  used  because  it  seemed  the  most 
fair  one,  analysis  of  the  uncorrected  scores  showed  that  there  was  no  clear  need  for  it.  The  overall  scorings  pattern 
remained  the  same  but  with  slightly  lower  mean  scores.)  To  solve  the  problem  subjects  in  the  Negative  Affect 
condition  generated  a mean  number  of  4.24  (5D=1.63)  solutions  and  subject  in  the  Positive  Affect  condition  a mean 
number  of  4.32  {SD=\.91)  solutions.  A t test  indicated  no  significant  differences  between  the  two  conditions 
(t[99]=-0.24,p=0.8 15,  two-tailed). 

Discussion 

The  purpose  of  the  current  study  was  to  explore  whether  the  satisficing/optimizing  model  of  Vosburg 
(1998)  could  be  verified  when  both  video,  as  mood  induction  technique,  and  two  creative  problem-solving  tasks  (a 
divergent  thinking  and  an  insight  task)  were  embedded  within  a Web-based  environment.  Table  1 presented  an 
overview  of  the  expected  effects.  The  results  show  that  the  two  affective  video  clips  were  effective  in  inducing  a 
positive  and  a negative  affective  state  in  the  subjects.  This  finding  replicated  the  results  of  a preliminary  experiment 
(Verleur  and  Verhagen,  2001).  The  data,  however,  do  not  confirm  the  satisficing/optimizing  model  nor  did  they 
replicate  findings  from  comparable  studies  that  were  conducted  in  more  traditional  settings. 

A first  question  that  occurs  is  whether  the  adaptation  to  the  Web  environment  or  the  changes  in  the 
methodology  may  have  prevented  the  replication  of  the  earlier  results.  Vosburg  (1998)  and  Kaufmann  and  Vosburg 
(1997)  used  paper-and-pencil  tests  to  administer  their  experiments.  It  cannot  be  guaranteed  that  filling  in  Web  forms 
gives  the  participants  equal  mental  room  for  playing  with  their  thoughts  during  the  different  tasks.  Kaufmann  and 
Vosburg  also  used  more  tasks.  Vosburg  (1998)  used  four  divergent  thinking  problems  and  Kaufmann  and  Vosburg 
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(1997,  Study  2)  used  two  insight  tasks.  They  based  their  conclusions  on  combined  scores  of  all  tasks  in  each  of  their 
experiments.  Vosburg  was  able  to  find  significant  effects  in  line  with  the  satisficing/optimizing  model.  Kaufmann 
and  Vosburg,  however,  did  not  find  a differential  effect  for  mood  on  task  performance,  which  is  in  line  with  our 
results.  As  Vosburg  (1998)  and  Kaufmann  and  Vosburg  (1997)  only  reported  the  combined  scores  of  the  tasks,  no 
data  are  available  to  analyze  whether  for  instance  the  use  of  the  class  problem  in  the  Web  environment  did  or  did  not 
equally  well  as  in  their  case.  Also  it  is  not  known  how  the  Class  problem  itself  performed  in  comparison  to  the  other 
divergent  thinking  tasks  used  by  Vosburg. 

The  choice  to  limit  the  number  of  tasks  to  one  insight  task  and  one  divergent  thinking  task  was  made  in 
order  to  make  the  experiment  manageable.  This  was  considered  to  be  feasible  because  it  was  expected  that  the 
difference  between  a well -chosen  insight  and  a well-chosen  divergent  thinking  task  should  be  observed  if  the 
satisficing/optimizing  model  is  valid.  This  expectation  was  supported  by  the  fact  that  the  present  study  used  the 
described  video-based  mood  induction  technique  (which  appeared  to  change  mood  significantly)  while  the  subjects 
in  Vosburg's  study  were  not  brought  into  the  different  affective  states  by  a mood  induction  technique,  but  that  the 
actual  affect  state  'on  arrival'  was  measured  and  taken  as  the  independent  variable.  Still  she  found  significant  results 
to  support  her  model.  A critical  question  is  whether  the  tasks  were  indeed  ‘well-chosen’.  As  there  are  no  data  about 
these  tasks  from  the  other  authors  and  there  was  also  no  control  condition,  this  is  difficult  to  know. 

One  possibility  is  that  the  tasks  were  not  so  different  after  all.  In  the  Method  part  of  this  paper  it  was 
explained  that  the  task  instruction  for  the  Class  problem  was  reformulated  to  make  sure  that  students  would  generate 
'different  solutions'  instead  of  a step-wise  procedure.  This  might  have  triggered  the  participants  to  look  more  at  the 
quality  of  their  solutions  to  make  sure  that  the  solutions  were  different  from  each  other,  which  could  have  changed 
the  perception  of  the  task  in  the  direction  of  an  optimizing  task.  However,  when  discussing  this  in  a debriefing 
session  with  a group  of  students  (where  about  30%  of  the  subjects  was  present),  the  students  reported  that  they  did 
realize  afterwards  that  the  task  evoked  a satisficing  strategy  in  them.  They  had  felt  that  they  could  end  the  task  after 
they  had  generated  'enough  solutions'.  Other  reasons  for  stopping  to  work  on  the  task  were  that  they  'run  out  of 
solutions'  or  'they  run  out  of  time'.  Although  the  instruction  was  not  literally  saying  to  'generate  as  much  solutions  as 
possible',  almost  all  students  did  understand  that  this  was  the  objective  for  the  task.  In  all  it  seems  that  the  task 
instruction  was  perceived  in  the  same  way  as  in  Vosburg's  study. 

A further  uncertainty  is  that  although  the  video  clips  were  able  to  evoke  the  different  affective  states  in  the 
subjects,  no  evidence  was  collected  that  this  evoked  state  was  continued  during  the  task  performance  and  thus 
remained  sufficiently  effective. 

The  uncertainties  about  the  applied  methods  make  that  conclusions  about  the  validity  of  the 
satisficing/optimizing  model  cannot  be  drawn.  There  are,  however,  a few  reasons  why  doubts  raise  about  this 
validity. 

In  the  study  of  Isen  et  al.  (1987,  Study  2)  subjects  were  presented  two  types  of  tasks,  one  of  them  being  an 
insight  task  (Duncker  Candle  task).  In  that  study  the  subjects  in  the  positive  mood  performed  better  than  the  subjects 
in  the  negative  mood.  Vosburg  (1998)  gives  a possible  explanation  for  this  effect:  The  Candle  task  was  performed 
using  real  attributes,  so  it  was  not  a paper-and-pencil  task.  By  physically  working  on  the  task  the  subjects  would 
receive  feedback  on  the  correctness  of  their  problem  solving  attempts.  This  type  of  feedback  might  have  yielded 
direct  information  on  how  to  proceed  for  finding  the  solution.  This  fact  may  explain  why  task  performance  differed 
from  paper-and-pencil  insight  tasks.  In  the  study  by  Isen  et  al.  there  were  four  conditions:  A positive  affect,  a 
negative  affect,  a neutral  affect  and  a control  condition.  A remarkable  finding  of  their  study  was  that  the  positive 
affect  condition  did  best,  than  the  negative  affect  condition,  followed  by  the  neutral  affect  condition.  The  control 
condition  did  worst.  Kaufmann  and  Vosburg  (1997)  put  forward  that  this  could  mean  that  there  is  a more  general 
performance-facilitating  factor  at  work  as  a function  of  mood  manipulations  prior  to  task  performance.  This  might 
suggest  that  there  is  a main  effect  for  mood  manipulation.  As  the  present  study  was  carried  out  without  a control 
condition,  no  data  are  available  to  analyze  whether  such  a general  effect  occurred. 

Kaufmann  and  Vosburg  (1997,  Study  2)  used  the  Two  String  problem  in  combination  with  another  insight 
problem  (Hatrack  problem)  in  a paper-and-pencil  version.  In  their  study  the  tools  that  were  available  to  solve  the 
Two  String  problem  were  a cup,  a screwdriver  and  a box  of  pins.  They  scored  the  tasks  according  to  a strict 
solved/unsolved  criterion,  where  one  point  was  given  for  a true  solution,  and  0 for  no  solution  or  quasi-solutions,  not 
fulfilling  all  solution  criteria.  In  the  present  study  only  one  of  the  tasks  was  used  and  for  which  the  scoring  system  of 
Martinsen  (1993)  was  followed.  The  same  visual  representation  of  the  problem  (with  adaptation  for  the  tools)  and 
instruction  was  used  as  in  Kaufmann's  and  Vosburg's  study.  In  general  more  subjects  (45%)  were  able  to  solve  the 
Two  String  problem  in  the  Kaufmann  and  Vosburg  study  compared  to  our  study  (19.8%).  This  might  have  been 
related  to  the  other  tools  used.  For  the  combined  insight  tasks  they  did  not  found  a significant  effect  of  affective  state 
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on  problem  solving  which  is  in  line  with  the  findings  in  the  present  study,  but  in  contradiction  with  the 
satisficing/optimizing  model. 

A different  issue  is  that  that  Kaufmann  and  Vosburg  did  find  a significant  effect  of  affective  state  on  the 
time  spend  to  solve  the  problem  (latency  rates)  or  'solution  ease'.  A negative  affect  state  produced  the  most  superior 
performance  (faster)  and  the  positive  affect  state  the  most  inferior  performance  (slower).  ‘Time’  was  not  used  as  a 
dependent  variable  in  this  study,  but  may  be  used  in  further  work. 

In  all,  the  experiment  was  not  able  to  confirm  the  satisficing/optimizing  model.  Further  work  has  to  be  done 
to  get  a better  understanding  of  the  relationship  between  video-induced  affective  states  and  problem  solving.  For 
subsequent  research  it  is  recommended  that:  (a)  the  research  methods  control  for  the  main  effect  of  mood  induction 
by  means  of  a no-treatment  control  condition;  (b)  possible  decay  of  induced  mood  during  the  experiments  is 
monitored  (at  least  by  post  testing  mood  state  after  the  task  performance);  (c)  several  (in  stead  of  one)  tasks  are  used 
for  each  factor  (insight  or  divergent  thinking)  to  middle  out  the  noise  that  is  caused  by  inevitable  task  imperfections; 
and  (d)  the  tasks  as  well  as  the  related  instructions  are  extensively  pre-tested  to  assure  their  validity  and  reliability. 
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Critical  thinking  and  discourse:  defining  the  concept 

Before  we  start  this  discussion  of  critical  thinking  it  is  useful  to  mention  that  the  authors  of  this  paper  see 
critical  thinking  as  a disciplined  manner  of  thought  (Scriven  & Paul,  2000)  and  that  a person  uses  to  assess  the 
validity  of  something  (statements,  stories,  arguments,  research,  etc.).  Critical  thinking  involves  things  such  as  asking 
questions (Facione,  1998),  defining  a problem,  examining  evidence(Facione,  1998),  analyzing  assumptions  and 
biases,  avoiding  over-simplification,  reflecting  on  other  interpretations,  and  tolerating  ambiguity  (Fisher,  2001). 

The  role  of  critical  thinking  and  discourse  in  education  has  been  recognized  through  the  centuries.  In  fact,  it 
was  Socrates  who,  through  discourse,  already  challenged  his  students  to  become  critical  thinkers  (Facione,  1998).  As 
such,  Socrates  encouraged  his  students  to  become  competent  in  knowing  how  to  formulate  and  ask  questions  and 
how  to  develop  a thorough  and  profound  understanding  of  the  issues  being  discussed  and  investigated. 

Critical  thinking  underlies  reading,  writing,  speaking  and  listening  (the  basic  elements  of  communication). 
When  discussing  critical  thinking  and  discourse  we  assume  that  these  basic  elements  of  communication  play  an 
important  role  in  all  levels  of  education,  and  particularly  in  graduate  level  education.  We  furthermore  assume  that 
critical  thinking  and  discourse  should  create  new  understandings  among  learners,  and  that  being  able  to  think 
critically  is  a “must”  in  a continuously  changing  world  where  being  able  to  adapt  to  a changing  environment  is  of 
utmost  importance(Facione,  1998). 

It  is  important  to  note  that  critical  thinking  includes  the  willingness  to  change  one’s  point  of  view  as  a 
result  of  examining  and  re-examining  ideas  and  facts  that  may  seem  obvious.  And,  even  more  so,  critical  thinking 
not  only  takes  time  and  effort,  it  also  includes  the  willingness  to  say  the  three  words  that  are  not  so  easy  to  use:  “ I 
don’t  know”. 

Some  findings  about  critical  thinking  in  traditional  university  courses 

There  are  a few  questions  to  ask  when  thinking  about  critical  thinking.  The  first  one  is  a very  basic  one: 

Are  we,  as  students  and  instructors  (as  learners)  helping  our  students  to  become  critical  thinkers?  One  group  of 
researchers  asked  this  question  in  their  research  project  in  38  public  and  28  private  universities  in  the  State  of 
California  (Paul,  Elder,  & Bartell,  1997).  They  assessed  current  teaching  practices  and  investigated  the  knowledge 
of  critical  thinking  among  faculty,  while  they  also  identified  exemplary  practices  in  teaching  critical  thinking.  The 
findings  were  not  very  encouraging. 

Although  almost  90%  of  the  instructors  claimed  that  critical  thinking  was  the  primary  objective  of  their 
instruction,  only  19%  of  the  instructors  could  give  a clear  explanation  of  critical  thinking,  while  only  9%  could 
clearly  differentiate  between  an  assumption  and  an  inference;  only  4%  could  differentiate  between  an  inference  and 
an  implication.  Although  67%  of  the  participating  instructors  said  that  their  concept  of  critical  thinking  is  largely 
explicit  in  their  thinking,  only  19%  could  elaborate  on  their  concept  of  critical  thinking(Paul  et  ah,  1997).  These  are 
interesting  findings  that  emphasize  the  need  for  more  profound  reflection  on  critical  thinking. 
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One  thing  is  sure  and  that  is  that  learning  environments  must  be  designed  in  such  a way  that  they  promote 
critical  thinking(Facione,  1998).  In  addition,  instructors  should  have  at  least  a basic  knowledge  of  the  concept  of 
critical  thinking  and  model  critical  thinking  in  their  own  instruction. 

We  should,  however,  also  mention  that  there  are  frequently  discrepancies  that  discourage  critical  thinking. 
We  often  see  that  university  students  are  supposed  to  develop  their  own  point  of  view,  have  to  avoid  being  spoon- 
fed, have  to  be  critical  thinkers,  but  also  have  to  adhere  to  the  system  by  producing  the  “right”  answers. 


Distance  Education  and  Critical  Thinking  and  Discourse 

In  recent  years  distance  education  has  become  an  increasingly  prominent  aspect  of  learning  and  instruction. 
There  are,  however,  fundamental  differences  in  the  challenges  and  opportunities  distance  education  offers  when 
compared  to  traditional  education.  Distance  education  has  the  last  quarter  of  the  twentieth  century  shown 
characteristics  of  an  industrialized  process  of  learning/teaching  and  frequently  does  not  really  allow  students  to 
use/apply  knowledge  in  a variety  of  ways.  Distance  education  organizations  are  still  in  the  process  of  moving  from 
Fordist  organizations  (uniform  products,  bureaucratic  processes)  to  Post-Fordist  organizations  (tailored  products, 
decentralized  approaches,  empowered  students  and  staff,  new  ways  of  doing  things,  being  dynamic,  moving  fast  and 
being  ready  for  change). 

If  we  look  at  the  following  important  areas  of  critical  thinking  a number  of  questions  are  justified  on  how  to 
include  these  areas  in  distance  education  environments: 

• Creativity  (distance  education  has  a tendency  to  adhere  to  an  industrialized  process) 

• Impact  of  language  (lack  of  attention  for  specific  communication  skills  needed  in  distance  education) 

• Decision  making  (curriculum  and  courses  in  distance  learning  are  often  very  much  prescribed  - no 
electives) 

• Meta-cognition  (distance  education  courses  tend  to  focus  on  content,  not  on  learning  processes  and  meta- 
learning) 

• Collaborative  learning  and  problem  solving. 

The  latter  area  - i.e.  collaborative  learning  and  problem  solving  - may  have  been  more  effectively 
developed  in  distance  education  than  in  traditional  education  and  may  thus  be  used  as  an  important  “carrier”  for 
critical  thinking  processes. 

There  are  a number  of  ways  to  use  collaboration  as  a way  of  promoting  critical  thinking: 

• Case-based  reasoning  (partner  activities:  share,  check,  review,  tell  and  retell  ideas) 

• Critiques,  rebuttals  around  topics  posted 

• Mock  trials,  debates,  posted  arguments  with  logic  delineated 

• Graphic  organizers,  flowcharts,  decision-making  trees,  concept  mapping.  (McVay  Lynch,  2002) 

Collaboration  in  distance  education  is  fostered  by  the  following  important  factors: 

• Increased  equity  among  participants  and  proven  high  participation  rates 

• Interactive  technologies,  growth  of  the  Internet,  the  potential  of  the  WWW. 

• Better  possibilities  for  students  to  explore  their  potential  as  thinkers  and  conveyers  of  ideas  through 
improved  access  to  (re)sources,  such  as  knowledge-based  communities. 

In  addition,  distance  students  have  “real-world“  experiences,  while  many  face-to-face  students  have  “real-world- 
like’’  experiences. 

To  foster  critical  thinking  and  discourse  in  education  we  suggest  that  first  of  all  the  instructional  design  of 
the  course  includes  critical  thinking  strategies,  such  as  clarity  of  objectives  and  of  the  expectations,  and  a Socratic 
perspective  in  encouraging  and  offering  discourse) ADEC,  2001).  Furthermore  critical  thinking  should  not  be 
something  that  “suddenly”  appears  and  is  expected  to  take  place  at  graduate  level  courses.  It  should  be  developed  at 
a much  earlier  level,  for  instance  through  philosophy  and  humanities  courses,  and  continue  to  be  part  of  the  learning 
and  teaching  process,  whereby  the  instructors  not  only  actively  encourage  critical  thinking,  but  also,  through 
dialogue  and  discussion  demonstrate  their  being  critical  thinkers. 


441 


Cultural  dynamics  and  discourse 

Students  in  traditional  classroom-based  learning  environments  may  come  from  a variety  of  different 
backgrounds  in  terms  of  their  social  culture  and  their  learning  culture.  However,  they  exercise  and  develop  their 
academic  growth  within  a fairly  fixed  community  operating  within  the  culture  and  values  prescribed  by  the 
institution  that  they  are  attending.  This  differs  significantly  from  distance  learning  environments,  in  which  learners 
continue  to  operate  within  their  own  social  and  cultural  contexts  and  thus  more  readily  integrate  this  familiar  context 
within  a distance  learning  environment.  It  is  likely  that  critical  discourse  and  thought  in  such  distance  learning 
contexts  is  fundamentally  different  in  nature  from  classroom-based  contexts.  This  notion  should  be  explored  further 
through  collaborative  efforts  of  learning  specialists  and  practitioners  in  both  traditional  and  distance  education 
settings. 

Final  considerations 

Research  in  educational  design  presently  argues  in  strong  support  of  the  notion  that  essential  skills  for 
negotiating  real  life  complexity  cannot  be  developed  in  isolation  from  the  authentic  context  in  which  such  skills  are 
to  be  applied.  Current  research  supports  the  notion  that  discovery  oriented  learning  environments  are  essential  for 
the  development  and  effective  transfer  of  higher  order  skills  applied  in  critical  thinking,  discourse  and  problem 
solving.  Since  distance  learning  environments  differ  from  classroom-based  environments  in  ways  that  are  central  to 
the  notion  of  “authentic  practice’’  the  strengths  and  weaknesses  of  each  of  these  environments  for  developing  and 
applying  critical  thinking  and  discourse  must  be  considered. 

Finally,  distance  education  is  in  an  excellent  position  to  explore  how  different  media  affect  the  learner’s 
cognitive  processes  in  different  ways.  In  considering  the  cognitive  processes  essential  to  higher  order  cognitive 
functioning,  the  effect  of  differing  media  on  the  development  of  such  skills  should  be  explored  further.  Such  inquiry 
must  seek  to  address  the  fundamental  goals  of  modern  educational  design,  in  which  the  primary  effort  is  to  develop 
individuals  capable  of  critically  interacting  with  a complex  and  changing  world,  and  capable  of  adapting  their 
learning  and  cognitive  processes  as  their  realities  change  and  evolve(Facione,  1998). 
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Abstract 

This  paper  discusses  the  present  situation  of  educational  technology  in  People’s  Republic  of  China  focusing 
on  the  problems  and  solutions  from  leadership  perspective.  The  problems  perceived  are  synthes  ized  from  the 
literature  review,  document  analysis,  and  the  personal  discussions  and  correspondences  with  those  people  working 
in  the  field  of  educational  technology.  There  are  four  parts  in  this  paper:  (a)  Defining  educational  technology  and 
educational  leadership,  (b)  problems  identified,  (c)  proposed  solutions,  and  (d)  conclusion.  This  paper  is  based  on  a 
pilot  result  of  an  on-going  research  in  Sichuan  Province,  P.  R.  China. 

Introduction 

Since  beginning  of  90s,  educational  technology  in  China  has  experienced  a great  change.  It  has  developed 
very  fast  from  the  use  of  simple  audio-visual  technology  to  application  of  complex  modern  communication 
technology  of  satellite  and  internets.  Its  great  potentials  in  improving  education  quality,  education  effectiveness  and 
education  efficiency  have  been  recognized.  It  begins  to  step  out  of  the  role  of  assisting  teaching  and  instruction  in 
education  to  functions  as  a new  critical  force  to  improve  education  practice,  to  increase  education  quality,  and  to 
implement  education  reform. 

To  further  unleash  these  great  potentials,  and  to  advocate  sound  practice  of  educational  technology,  authors 
of  this  article  initiated  a research  project  to  examine  the  perception  of  educational  technology  of  those  leaders  who 
worked  in  the  front  line  of  educational  technology.  The  research  site  is  in  Sichuan  Province,  which  is  in  the 
southwest  part  of  P.  R,  China.  The  province  has  over  100  million  people  and  is  usually  regarded  as  a very  typical 
province  in  China  in  terms  of  education  and  economic  development.  Based  on  the  pilot  study  results  and  together 
with  observations  made  during  past  few  years  in  our  work,  problems  in  current  educational  technology  practice  are 
synthesized  and  possible  solutions  are  proposed  from  leaders’  perspective.  The  problems  synthesized  are  also  typical 
in  many  other  provinces,  especially  those  in  the  southwest  part  of  China  (Wang,  2001). 

There  are  four  parts  in  this  paper:  (a)  Defining  educational  technology  and  educational  leadership,  (b) 
problems  identified,  (c)  proposed  solutions,  and  (d)  conclusion. 

Defining  educational  technology  and  educational  leadership 

Educational  technology 

Educational  technology,  sometimes  known  as  instructional  technology,  is  concerned  with  the  improvement 
of  educational  programs  and  educational  systems  through  the  application  of  technology.  The  representative 
definition  of  instructional  technology  prepared  by  the  AECT  Definitions  and  Terminology  Committee  (2000)  well 
defines  concept  of  educational  technology. 

Instructional  Technology  is  the  theory  and  practice  of  design,  development,  utilization, 

management,  and  evaluation  of  processes  and  resources  for  learning.  ...  The  purpose  of 

instructional  technology  is  to  affect  and  effect  learning  (Seels  & Richey,  1994,  pp.  1-9) 

(online:  http://www.aect.org/Affiliates/National/Standards.pdf) 

As  a relatively  new  field  of  specialization,  there  is  no  clear  cut  of  its  boundaries  in  the  field  of  educational 
technology.  It  draws  upon  the  essence  mainly  from  the  fields  of  psychology,  system  theory,  communication, 
leadership  and  management,  and  science  and  technology,  in  order  to  improve  human  learning  and  human 
performance  (Peck,  2000).  Thus,  its  theoretical  research  is  interdisciplinary  one,  encompassing  learning  and 
cognition;  instructional  design  theories  and  models;  instructional  strategies  and  tactics;  instructional  media. 
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educational  systems  design,  development  and  evaluation,  system  change  and  leadership,  etc.  In  short.  Educational 
technology  is  about  how  to  improve  human  learning  and  performance  in  diverse  contexts  in  an  efficient  and  cost 
effective  fashion. 

Educational  leadership 

People  define  it  from  different  perspectives.  Owens  (1995)  defines  leadership  as  a group  process  that 
influences  the  actions  of  other  participants.  Keith  and  Girling  define  leadership  as  a relationship  between  a leader 
and  followers  involving  power,  vision,  and  influence  (1991,  p.  57).  Wills  (1994)  emphasizes  that  the  leadership  must 
contain  three  elements:  a competent  leader,  willing  and  able  followers,  and  worthwhile  and  shared  goals.  Maurer 
and  Davidson  assert  “leadership  is  ambiguous,  amorphous,  contextual,  and  idiosyncratic.  Leaders  must  create  their 
own  definition  of  leadership  for  themselves  that  are  consis  tent  with  their  values  and  beliefs”  (1998,  p.  16). 

Our  definition  of  leadership  used  in  this  paper  is  the  process  of  decision  making  and  problem  solving  in 
achieving  organizational  objectives  and  goals,  realizing  organizational  vision,  and  accomplishing  organizational 
mission  by  guiding,  directing,  motivating  or  influencing  people  and  organizations. 


Problems  identified 

Based  on  the  pilot  study  result  and  together  with  observations  made  during  past  few  years  in  our  work, 
following  problems  are  identified  in  the  field  of  educational  technology  in  China.  They  are  (1)  misconception  of 
educational  technology,  (2)  inappropriate  mission  for  the  educational  technology  units  in  respective  educational 
institutions,  (3)  poor  management  and  utilization  of  limited  resources,  and  (4)  administrative  problem. 

Misconception  of  educational  technology 

Educational  technology  is  still  undergoing  through  the  process  of  development.  In  spite  of  its  merits,  there  are  a 
lot  of  problems  need  to  be  solved.  These  problems  in  the  current  practice  of  educational  technology  in  China  are  in 
one  way  or  another  related  to  people’s  fundamental  perception  and  understanding  of  what  educational  technology  is. 
Since  people’s  perception  and  understanding  of  educational  technology  direct  influence  their  decision  making  in 
practice  of  educational  technology,  many  of  the  problems  actually  stem  from  the  misconception  of  educational 
technology.  The  misconception  of  Educational  technology  is  in  two  folds.  Eirst,  educational  technology  is  the  “hard 
and  solid”  product  of  modern  science  and  technology.  The  second  one  is  that  educational  technology  is  privilege 
reserved  for  those  specialized  professionals.  Only  those  well  trained  professionals  know  applications  of  educational 
technology.  Both  of  perceptions  of  educational  technology  are  very  similar  to  the  idea  that  regards  educational 
technology  as  “hardware”  only. 

Inappropriate  mission 

With  fast  development  of  science  and  technology,  many  educational  organizations  and  institutions  have  set 
up  a educational  technology  unit  responsible  for  managing  the  equipment  and  facilities  of  educational  technology. 
The  unit  varies  in  the  number  of  personnel  from  45  in  a large  university  to  1 person  in  a local  school. 

Unfortunately  in  our  research  results  show  that  many  of  these  educational  technology  units  do  not  have  an 
appropriate  mission  to  accomplish.  The  inappropriate  mission  statement  of  educational  technology  office  from  a 
large  university  in  Sichuan  Province  is  typical  among  these  units: 

“Our  mission  is  to  serve  for  students,  faculty,  and  staff  of  the  university  by 

• Managing  and  maintaining  the  educational  equipment  and  facilities; 

• Providing  training  on  using  educational  equipment  and  facilities; 

• Providing  in-time  technical  support.” 

With  such  a narrow-minded  and  inappropriate  mission  statement,  the  university  center  of  educational 
technology  is  turned  into  a rear  service  department.  It  is  not  an  educational  frontier  force  any  more.  Although  many 
of  these  educational  technology  units  also  provide  training  to  university  faculty  and  school  teachers,  the  training 
programs  are  often  focused  on  the  usage  of  equipment  and  facilities.  They  are  for  the  protection  of  the  equipment 
and  facilities  rather  than  for  the  active  and  creative  use  of  them. 
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Poor  management  of  limited  resources 

The  inappropriate  mission  of  educational  technology  units  leads  to  another  common  problem:  the  poor 
management  of  limited  resources.  The  poor  management  of  resources  includes  the  human  resource,  equipment  and 
facilities,  and  finance. 

Because  the  main  task  of  these  educational  technology  units  is  to  manage  and  maintain  the  equipment  and 
facilities,  the  most  personnel  in  these  units  are  engineers  and  technicians.  They  are  experts  in  the  equipment  and 
facilities  in  terms  of  their  usage  rather  than  their  uses.  The  personnel  who  are  specialized  in  application  of 
educational  technology  are  rarely  found  in  these  units.  This  problem  in  human  resource  management  makes  many 
educational  technology  units  incompetent  to  promote  the  sound  practice  of  educational  technology. 

For  the  same  reason,  managing  and  maintaining  the  equipment  and  facilities,  the  equipment  and  facilities 
are  tend  to  be  tightly  controlled  by  those  educational  technology  units.  The  equipment  and  facilities,  especially  those 
new  and  high-priced  ones  are  not  easily  accessible.  In  fact,  many  teachers  would  rather  not  use  it  because  of 
troublesome  access  process.  Thus,  the  educational  equipment  and  facilities  are  really  in  the  hands  of  technicians 
rather  than  in  the  hand  of  students  and  teachers. 

Another  instance  of  poor  management  of  limited  resource  is  of  finance.  In  the  development  of  educational 
technology,  many  educational  technology  units  look  for  huge  project  that  could  help  them  to  acquire  huge  funds. 
Usually,  the  educational  technology  units  are  willing  to  spend  lots  of  money  from  their  limited  resources  on  fancy 
labs,  to  spend  thousands  dollars  to  update  a computer,  yet  very  reluctant  to  spend  anything  on  simple  overhead 
projectors.  Their  finance  is  not  prioritized  according  to  the  actual  needs  of  educational  technology  from  students  and 
teachers.  For  example,  there  are  the  project  of  inter-communication  of  “Earth-Net  (Internet)  and  Sky-Net  (satellite- 
net)”,  the  project  of  inter-university  communication  stations,  and  broadband  project  within  each  university  going  on 
in  Sichuan  Province.  These  projects  require  huge  investments  and  are  usually  designed,  developed  and  managed  on 
a project  scope.  There  is  no  systemic  analysis  of  its  needs.  No  wonder  some  one  comments  on  these  projects  saying 
that  these  projects  are  great  and  necessary,  but  the  equipment  of  its  end  users  are  outdated  and  they  could  hardly 
benefit  from  the  newly  built  up  facilities.  This  problem  needs  to  be  addressed  right  away.  This  is  also  closely  related 
to  the  administration  problem  that  looks  only  at  the  numbers  and  fancy  equipment  in  evaluation  of  those  educational 
technology  units. 

In  fact,  these  three  problems  relating  to  the  management  waste  a lot  of  limited  resource  in  educational 
technology  practice. 

Administrative  problem 

The  perceived  problems  in  the  practice  of  educational  technology  do  not  stand  in  isolation.  They  are,  as 
mentioned  above,  closely  related  to  the  problem  in  the  whole  educational  system.  In  terms  of  administration, 
educational  technology  is  not  well  integrated  in  their  respective  educational  system  such  as  universities  or  schools. 
As  a matter  of  fact,  the  education  administrations  very  often  misguide  the  practice  of  educational  technology.  This  is 
especially  true  in  evaluation  and  financial  planning.  The  educational  leaders  tend  to  look  at  numbers  of  equipment 
and  facilities  instead  of  the  actual  effects  of  their  uses  in  evaluation  of  educational  technology  units. 

Second,  there  are  almost  no  incentives  and  rewards  for  promoting  sound  practice  of  educational  technology 
at  any  level  of  the  educational  systems.  Incentives  and  rewards  are  provided  for  managing  and  maintaining  the 
equipment  and  facilities.  There  is  no  administrative  effort  that  concretely  recognizes  that  the  educational  technology 
is  the  frontier  force  of  educational  practice  and  educational  reform. 

Although  many  educational  leaders  themselves  will  not  deny  the  idea  that  educational  technology  has  great 
potentials  in  improving  educational  practice  and  educational  reform,  the  actual  practice  of  their  leadership  shows 
that  they  fail  to  see  or  realize  the  great  potentials  of  educational  technology.  Under  their  leadership,  the  system  fails 
to  build  in  itself  the  mechanism  to  promote  the  sound  practice  of  educational  technology  and  actualize  its  potentials. 
Under  such  circumstances,  we  can  hardly  expect  that  educational  technology  units  function  properly  and  effectively 
in  accomplish  its  appropriate  mission. 

Possible  solutions  proposed 

The  following  solutions  are  based  on  our  personal  understanding  of  the  problems,  and  change  model  offered  by 
Roger  (1995)  and  Engineering  Worth  Performance  offered  by  Gilbert  (1996).  The  possible  solutions  to  the  problems 
proposed  are  only  initial  attempts.  Eor  this  reason,  I name  this  section  as  possible  solutions  proposed.  The  proposed 
solutions  include  (1)  system  approach  towards  system  change,  (2)  top-down  and  bottom-up  strategies  of  change,  and 
(3)  sustained  efforts. 
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Proposed  solution  1:  System  approach  towards  system  change 

The  system  approach  refers  to  the  way  that  takes  every  aspect  of  the  problems  identified  into  consideration  and 
tackles  them  holistically  in  problem-solving  process.  In  our  cases,  to  rectify  the  problems  in  educational  technology 
practice  is  not  only  to  fix  the  problems  themselves  but  to  improve  the  whole  system  of  educational  technology  and 
maybe  even  to  improve  its  superior  organization  by  fixing  those  problems.  This  approach  should  be  differentiated 
from  systematical  approach.  The  systematical  approach  is  linear,  step-by-step  approach  while  the  systemic  is 
integrated,  holistic  approach.  (Banathy,  1996). 

Change,  according  to  Fullan  and  Stiegelbauer  (1991),  Reigeluth  and  Garnfikel  (1994),  Roger  (1995),  and 
Ellsworth  (2000),  is  a process  rather  than  an  event.  The  purpose  of  organizational  change  is  to  achieve  the 
organizational  goals  more  efficiently  and  effectively.  This  requires  a systemic  approach  in  dealing  with  change.  In 
other  words,  the  proposed  solutions  here  should  not  be  considered  as  isolated  measures  but  many  of  integrated 
efforts  in  the  problem-solving  process  to  promote  the  sound  practice  of  educational  technology  in  education. 

Specifically,  the  systems  approach  recommended  should  have  three  main  characteristics:  First,  the  effort  to 
change  in  educational  technology  must  be  carried  out  at  all  different  levels  of  the  educational  systems  (from  the 
provincial  bureau  of  education  to  local  school)  in  order  to  be  effective.  Second,  within  each  educational  technology 
unit,  the  consideration  to  implement  the  change  should  include  the  aspects  at  philosophical,  culture,  policy,  strategic, 
tactics  and  logistics  levels  as  suggested  by  Gilbert  (1996).  The  effort  should  cover  all  the  cells  in  the  following  table 
of  change  effort.  Programs  to  facilitate  this  change  should  be  offered  to  people  involved  so  that  they  can  reach  a 
consensus  on  the  mission  of  educational  technology  unit  and  their  individual  responsibilities  in  accomplishing  it. 


Provincial 
Bureau  of  Ed. 

Municipal 
Bureau  of 
Ed./University 

County 
Commission 
of  Ed. 

Town  & District 
Committee  of 
Ed. 

Local  School 
Administration 

Philosophic  Levels 

Cultural  Levels 

Mission/Vision  Levels 

Strategic  Levels 

Tactic  Levels 

Logistic  Levels 

Because  this  system  change  affects  the  majority  people  of  educational  technology  units  including  their  job 
security  and  role  of  life,  another  characteristics  the  system  approach  should  have  is  dynamic  balance  toward  the 
change.  In  many  situations,  change  is  not  one  way  or  the  other.  It  is  a matter  of  degree  and  extent  of  effort  to 
maintain  well-balanced  progress.  At  least  following  list  of  issues  of  balance  should  be  considered  in  the  system 
change  process.  They  are  not  exhaustive  but  good  to  start  with.  Only  the  balanced  approach  can  bring  the  desired 
changes  effectively  in  our  case. 


Organization  goals 
Organization  benefits 
Autocratic  leadership/  Control 
Provincial  level  of  change 
Resources  for  education 
Fast  radical  change 
Change  personnel 
Old  mission 


Balance  ^ 

Individuals’  goal 
Individuals’  benefits 
Democratic  leadership/  Empowerment 
Local  school  level  of  change 
Resource  for  change 
Maintain  status  quo 
Educating  personnel 
New  mission 


Proposed  solution  2:  Top-down  and  bottom-up  strategies  of  change 

Top-down  strategy  of  change  is  recommended  mainly  because  of  the  centralized  educational  system.  The 
educational  leaders  at  various  levels  can  impact  the  system  tremendously.  To  change  them  is  almost  equal  to  change 
the  whole  system.  Besides,  the  rooted  tradition  to  follow  the  leaders  will  spread  very  quickly  across  the  system  the 
correct  concept  of  educational  technology  if  the  educational  leader  at  a higher  level  initiated  the  change.  However, 
bottom-up  strategy  of  change  is  also  recommended  because  many  changes  have  to  be  implemented  at  very  local 
level. 

Specifically,  I recommend  to  initiate  the  change  by  educating  educational  leaders  about  educational  technology 
at  provincial,  municipal,  and  finally  at  county  levels.  There  are  two  reasons  behind  this  proposed  solution.  First, 
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although  the  educational  system  is  still  centralized  hierarchical  system,  it  is  changing  and  becoming  more  open, 
receptive,  and  dynamic  than  ever  before.  The  concepts  such  as  life-long  learning  are  widely  accepted  among  many 
educational  leaders  at  various  levels.  In  Sichuan  and  other  16  provinces  in  China,  all  directors  under  age  40  at 
provincial,  municipal,  and  county  levels  are  required  to  take  either  a leadership  or  management  certificate  program 
to  be  qualified  for  their  positions.  There  are  possibilities  to  educate  those  educational  leaders.  It  is  crucial  to  help 
those  leaders,  who  are  very  often  educational  policy  makers,  understand  correctly  about  educational  technology  and 
its  potential  power  to  improve  educational  practice.  Second,  by  educating  those  educational  leaders  in  the 
educational  system,  the  ideas  of  potentials  of  educational  technology  in  educational  practice  and  reform  are  more 
effectively  communicated. 

To  ensure  that  people  at  local  level  of  educational  technology  are  actively  involved  in  the  change  process,  more 
powers  should  be  given  to  the  local  educational  technology  unit.  These  people  are  working  in  the  front  line  of 
educational  technology  and  their  concerns  can  never  be  ignored.  In  many  cases,  we  found  that  the  really  needs  to 
change  starts  at  the  local  levels. 

Proposed  solution  3:  Sustained  efforts 

The  sustained  effort  of  change  includes  (1)  establishing  new  mission  for  educational  technology  units  at  all 
levels,  (2)  improving  the  management  of  educational  technology  units,  and  (3)  establishing  support  system. 

(1)  Establishing  new  mission  for  educational  technology  units 

This  proposed  solution  originates  from  Gilbert  engineering  worthy  performance  concept.  Gilbert  (1996)  asserts 
that  any  human  performance  achievement  should  be  measured  according  to  the  goal  of  the  organization  and  the 
organizational  goal  and  objectives  must  be  in  alignment  with  mission,  vision,  and  the  philosophical  values  of  the 
organization.  If  the  mission  of  an  organizational  unit  is  unclear  and  inappropriate,  it  is  very  like  to  run  off  the  track. 

The  new  mission  statement  for  the  educational  technology  unit  is  recommended  to  at  least  include  the 
following: 

• To  promote  the  active  and  creative  use  of  educational  technology  in  educational  practice; 

• To  design  and  develop  effective  and  efficient  instructional  programs  by  collaboration  with  teachers 

• To  conduct  research  relating  to  educational  technology  to  expand  the  knowledge  of  it  and  enhance  the 
ability  of  applying  those  knowledge  in  education. 

• To  provide  training  programs  on  the  uses  of  educational  technology 

• Manage  and  maintain  the  educational  technology  equipment  and  facilities  and  provide  technical  supports 

The  new  mission  statement  is  an  initial  attempt  to  turn  these  educational  technology  units  into  a frontier  force  in 

educational  practice  and  educational  reform.  It  is  an  effort  to  recognize  what  educational  technologist  can  do  in 
improving  educational  practice.  Educational  technology  unit  should  not  be  regarded  as  the  rear  service  department! 

(2)  Improving  the  management  of  educational  technology  units 

“ ‘Healthy’  institutions  are  ‘fit  for  purposes’;  in  other  words,  they  are  organized  to  ensure  that  their  goals  and 
purposes  are  achieved  in  the  most  effective  and  economical  manner  “(Bates,  2000,  p.36).  The  newly  established 
mission  should  rectify  the  original  purposes  of  the  educational  technology  units.  To  accomplish  the  newly 
established  mission,  the  management  of  educational  technology  units  has  to  be  improved.  This  improvement  of 
management  should  be  parallel  to  the  three  problems  identified  in  the  management  of  human  resource,  equipment 
and  facilities,  and  finance. 

For  the  management  of  human  resources,  the  human  structure  of  educational  technology  unit  needs  to  change. 
The  present  human  structure,  with  engineers  and  technicians  taking  up  65  ~ 85  % of  its  workforce,  cannot  be 
efficient  in  accomplishing  its  new  mission. 

The  practical  methods  dealing  with  this  problem  are  of  three.  The  first  is  to  recruit  educational  technologists  to 
work  in  these  units  so  that  it  has  capable  people  to  accomplish  its  new  mission.  The  second  is  to  invite  experienced 
teachers  and  university  faculty  to  work  with  those  engineers  and  technicians  in  educational  technology  unit.  This 
would  reinforce  its  human  resources  in  accomplishing  the  new  mission.  The  third  is  to  train  some  young  engineers 
and  technician  to  become  competent  educational  technologist  through  career  development  and  practice. 

In  management  of  equipment  and  technology,  to  make  all  the  equipment  and  facilities  accessible  to  the  students 
and  teachers  should  be  its  first  priority.  The  complicated  procedures  of  arranging  equipment  and  facilities  must  be 
simplified.  The  purpose  of  managing  educational  technology  equipment  and  facilities  is  to  maximize  the  use  of 
them.  I believe  that  inviting  faculty  to  work  in  the  educational  technology  unit  will  also  make  educational 
technology  equipment  and  facilities  more  easily  accessible  to  students  and  teachers. 

For  the  management  of  finance,  educational  technology  units  of  different  levels  should  conduct  thorough 
equipment  and  facility  need  analysis  before  purchasing  any  equipment  and  facilities.  The  practical  method  is  to 
organize  an  ad  hoc  committee  for  financial  management  at  various  levels  that  represent  all  the  parties  involved  in 
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the  development  of  educational  technology.  They  are  responsible  for  the  conducting  equipment  and  facilities  needs 
analysis  and  making  recommendations  for  educational  technology  unit  in  terms  of  financial  management  and 
equipment  purchase. 

(3)  Establishing  support  systems 

To  help  the  organization  to  go  through  the  change  process  smoothly  and  effectively,  supporting  systems  must 
be  built  to  ensure  the  sustained  efforts  in  change.  The  supporting  systems  should  at  least  include  incentive  and 
rewards,  evaluation,  collaboration,  and  resources  at  all  levels  from  provincial  bureau  of  education  to  the  local  school 
administrations. 

Establishing  a supporting  system  of  incentives  and  rewards  is  to  encourage  and  motivate  people  for  promoting 
the  sound  practice  of  educational  technology.  The  incentives  and  rewards  should  be  based  on  the  scientific 
evaluation  of  educational  technology.  This,  consequently,  involves  a system  of  evaluation.  The  evaluation  system 
should  support  the  change  efforts  by  not  only  looking  at  the  numbers  but  the  effects  of  educational  technology  in 
actual  practice. 

Establishing  a system  of  collaboration  across  all  levels  of  educational  technology  units  and  across  other 
departments  within  their  respective  schools  and  organizations  is  another  way  to  sustain  the  change.  This  ensures  the 
support  hierarchically  from  the  superior  organization  and  support  horizontally  from  other  departments  in  educational 
technology  practice. 

Establishing  a system  of  resources  is  another  way  to  sustain  the  efforts  in  implementing  changes.  The  resources 
include  human  resource,  financial  resource,  equipment  and  facility  resources,  etc.  All  these  supporting  systems  have 
to  work  together  to  sustain  the  efforts  in  implementing  the  change,  if  the  change  is  meant  to  be  effective  one. 


Conclusion 

Educational  technology  is  an  important  indicator  of  the  modernization  of  Chinese  education  (China 
Education  and  Research  Network,  2000).  The  government  as  well  as  educational  leaders  at  various  levels  has  come 
to  realize  the  important  roles  and  functions  of  educational  technology  in  educational  practice  and  reform.  However, 
this  is  far  from  enough. 

Change  is  concrete.  The  results  must  be  observable.  Change  is  complicated.  The  approach  must  be 
systemic  and  balanced.  Change  is  a difficult  process.  We  must  build  supporting  systems  to  sustain  the  efforts  to 
change.  Change  is  evitable  and  we  all  have  to  face  it!  There  are  much  more  that  need  to  be  done  but  we  must,  we 
can  and  we  will  implement  this  difficult  change! 
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Abstract 

This  study  examined  the  effects  of  three  concept  mapping  strategies  in  facilitating  achievement  of  different 
types  of  educational  objectives.  The  three  concept  mapping  strategies  were  concept  identifying,  proposition 
identifying  and  student  generated  concept  mapping.  The  instructional  material  was  a 2000- word  expository  text 
related  to  the  physiology  and  functioning  of  the  human  heart.  Instruction  was  presented  in  a web  based  environment. 
Achievement  was  measured  by  three  individual  and  a total  criterion  tests.  105  college  levels  students  were  randomly 
assigned  to  one  of  four  instructional  treatments.  Criterion  tests  were  administered  immediately  after  students 
interacted  with  their  respective  treatments.  One-way  analysis  of  variance  indicated  that  (a.)  specific  concept 
mapping  strategies  can  significantly  improve  student  achievement  of  specific  types  of  educational  objectives  and  (b) 
all  types  of  concept  mapping  strategies  are  not  equally  effective  in  facilitating  student  achievement. 

Introduction 

Concept  maps  are  most  commonly  defined  as  two  dimensional  diagrams  that  consist  of  concepts  or  nodes 
linked  by  labeled  lines  to  show  relationships  between  and  among  those  concepts.  The  concepts  are  usually  arranged 
hierarchically  with  the  most  inclusive,  general  concepts  appear  at  the  top  of  the  map  and  the  less  inclusive, 
subordinate  concepts  below  (Jonasson  & Grabowski,  1993;  Jonasson,  Beissner,  & Yacci,  1993;  Novak,  1990,  1998; 
Plotnick,  1997,  2001).  The  web-based  learning  environment  refers  to  those  learning  situations  where  instructions  are 
delivered  over  computer  network  and  displayed  by  web-browser  (Clark,  2000).  In  a web-based  learning 
environment,  concept  maps  have  been  used  for  course  site  navigation  (Gaines  & Shaw,  1995;  Tan,  2000),  as  a 
learning  tools  to  facilitate  learning  (Gaines  & Shaw,  1995;  Shortridge,  2001),  and  as  evaluation  tools  to  assess  the 
learning  (Tsai,  Lin,  & Yuan,  2001).  This  study  examines  the  use  of  varied  concept  maps  to  facilitate  student 
achievement  of  different  educational  objectives  in  a web  based  learning  environment. 

Studies  of  concept  map  as  a tool  to  facilitate  learning  are  extensive  and  diverse.  One  group  of  the  studies 
focuses  on  the  effects  of  concept  maps  used  in  different  formats.  Examples  include  the  instructor-provided  concept 
map  versus  student  completed  or  generated  concept  map  (Wang,  1995;  Smith  & Dwyer,  1995);  individual  concept 
mapping  versus  group  concept  mapping  (Brown,  2000);  and  hand  drawn  versus  computer  generated  concept  maps 
(Sturm,  1996).  The  second  group  of  studies  compares  and  contrasts  the  effects  of  concept  map  with  those  of  other 
learning  strategies.  Examples  for  this  group  of  studies  are  comparing  and  contrasting  concept  mapping  with  note- 
taking, outlining  (Keng,  1996),  Gowin’s  Vee  heuristic  diagramming  (Rumer,  1990;  Passmore,  1996),  post- 
questioning and  feedback  (Naidu,  1991).  The  third  group  of  concept  map  studies  focuses  on  effects  of  concept  maps 
when  they  are  applied  in  a specific  area  or  with  specific  group  of  people.  Examples  in  this  group  are  the  studies  of 
concept  mapping  in  science  classroom  (Green,  Ritchie,  & Volkl,  2000)  and  with  low-achieving  inner-city  seventh 
graders  (Guastello,  Beasley,  & Sinatra,  2000). 

Although  concept  maps  as  a tool  to  facilitate  learning  are  generally  effective  and  robust  (Winn  & Snyder, 
1996),  it  cannot  be  assumed  that  concept  map  works  equally  well  under  all  conditions  to  achieve  all  kinds  of 
learning  objectives.  Research  also  revealed  some  problems  in  the  use  of  concept  mapping.  Wandersee  (1990)  and 
Romance  & Vitale  (1999)  reported  that  students  had  problems  formulating  meaningful  propositions  and  efficiency 
of  student  generated  concept  mapping  was  generally  low.  Beissner,  Jonassen,  and  Grabowski  (1994)  pointed  out: 
“When  different  graphic  techniques  are  used  as  learning  tools,  different  cognitive  processes  are  elicited,  resulting  in 
different  learning  outcomes”  (p.20).  They  also  suggested  a match  between  the  desired  learning  outcomes  and 
characteristics  of  the  graphic  techniques  used  to  achieve  those  outcomes.  In  general,  when  different  learning 
objectives  are  to  be  achieved,  it  seems  reasonable  to  expect  that  those  types  of  concept  maps  which  instigate  deeper 
levels  of  cognitive  processing  would  be  most  effective  in  facilitating  higher  levels  of  learning.  Concept  map 
development  consists  of  three  important  elements:  the  concepts,  the  propositions  and  its  hierarchical  structure.  By 
controlling  and  manipulating  these  variables  in  concept  mapping,  it  is  hypothesized  that  the  different  levels  of 
cognitive  process  will  be  effected  and  therefore  instigate  different  levels  of  information  processing. 
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Statement  of  the  problem 

In  the  literature,  many  studies  indicate  that  concept  maps  as  a tool  to  facilitate  learning  are  effective  and 
robust . However,  there  is  limited  research  on  the  appropriate  use  of  concept  mapping  strategies  regarding  their 
effectiveness  in  facilitating  the  achievement  of  specific  learning  objectives.  This  study  attempts  to  examine  the 
instructional  effects  of  three  concept  mapping  strategies  used  to  facilitate  student  achievement  of  different 
educational  objectives. 

Procedure  and  treatments 

One  hundred  fifty  six  college  level  students  participated  in  this  study.  Extra  credit  points  were  awarded  for 
participation.  Students  were  randomly  assigned  to  one  of  the  four  treatment  groups.  One  week  prior  to  the  study,  six 
workshops  on  concept  mapping  were  offered  to  the  students  in  the  varied  treatment  groups.  Students  in  control 
group  received  the  same  workshop  about  concept  mapping  after  the  study.  Each  workshop  lasted  for  about  one  hour. 
Written  instruction  for  the  study  was  given  to  every  subject  when  they  came  to  the  lab.  Eollowing  the  instruction,  the 
subjects  browsed  the  web  course  site,  interacted  with  the  instructional  material  at  their  own  pace  to  understand  its 
contents  and  then  took  the  different  criterion  tests  (identification,  terminology,  and  comprehension)  online.  The 
instructional  material  used  in  this  study  was  a meaningful  2000- word  expository  text  describing  the  human  heart 
including  its  parts,  locations  and  functions  during  systolic  and  diastolic  phases.  The  material  was  developed  by 
Dwyer  and  Lamberski  (1977)  for  the  purposes  of  measuring  the  instructional  effect  of  different  independent 
variables  on  different  types  of  educational  objectives.  Eollowing  is  a description  of  each  treatment  in  this  study. 

Treatment  Group  1:  Control  Group 

Students  in  this  group  were  required  to  interact  with  the  instructional  material  at  their  own  pace  and  to  take 
the  online  criterion  tests  immediately  after  they  finished  the  instructional  materials  and  felt  that  they  understood  its 
contents. 

Treatment  Group  2:  Concept  Identifying  Mapping  Group 

Students  in  this  group  were  required  to  interact  with  the  instructional  material  at  their  own  pace  and  to  use 
concept  identifying  mapping  strategy  to  summarize  the  instructional  material.  They  took  the  online  criterion  tests 
after  they  finished  the  concept  mapping  activity.  Eigure  1 is  an  example  of  the  concept  identifying  mapping  activity 
in  Treatment  2. 
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FIGURE  1.  Sample  activity  of  concept  identifying  mapping  strategy 
Direction:  Please  complete  the  concept  map  by  filling  in  the  concepts  according  to  reading  Part  2 & 3. 


Concept  Map  of  Heart  Structure 


Treatment  3:  Proposition  Identifying  Mapping  Group 

Students  in  this  group  were  required  to  interact  with  the  instructional  material  at  their  own  pace  and  to  use 
proposition  identifying  mapping  strategy  to  summarize  the  instructional  material.  They  took  the  online  criterion  tests 
after  they  finished  the  concept  mapping  activity.  Figure  2 is  an  example  of  the  concept  identifying  mapping  activity 
in  Treatment  3. 


FIGURE  2.  Sample  activity  of  proposition  identifying  mapping  strategy 
Direction:  Please  complete  the  concept  map  by  filling  in  concepts  and  link  words  according  to  reading  Part  2 & 3 . 


Concept  Map  of  Heart  Structure 
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Treatment  4:  Student  Generated  Mapping  Group 

Students  in  this  group  were  required  to  interact  with  the  instructional  material  at  their  own  pace  and  were 
required  to  generate  their  concept  maps  summarizing  the  instructional  material  by  identifying  both  the  concepts  and 
propositions  in  the  instructional  material.  They  took  the  online  criterion  tests  after  they  finished  the  concept  mapping 
activity.  Figure  3 is  an  example  of  student  generated  concept  mapping  activity  in  Treatment  4. 

FIGURE  3.  Sample  activi  ty  of  student  generated  concept  mapping  strategy 
Direction:  Please  create  a concept  map  about  heart  structure  according  to  reading  Part  2 & 3 . 


Concept  Map  of  Heart  Structure 


Criterion  measures 

The  objective  of  each  criterion  test  was  as  follows:  (a)  identification  test  - to  measure  transfer  of  learning  (i.e., 
the  ability  to  identify  numbered  parts  of  the  heart  based  on  the  information  received  in  the  study),  (b)  terminology 
test  - to  evaluate  students’  knowledge  of  references  for  specific  symbols,  (c)  comprehension  test  - to  measure 
understanding  of  the  heart,  its  parts  and  functions,  and  (d)  total  criterion  test  - to  measure  the  students’  total 
understanding  of  the  concept  presented  (Roshan  & Dwyer,  1998).  The  following  description  of  the  criterion  tests 
used  in  this  study  was  adapted  from  Roshan  and  Dwyer  (1998)  and  Baker  and  Dwyer  (2000). 

Identification  Test 

This  test  of  20  item  multiple-choice  question  was  designed  to  measure  the  subjects’  the  ability  to  use  visual  cues 
to  discriminate  one  structure  of  the  heart  from  another  and  to  associate  specific  parts  of  the  heart  with  their  proper 
names.  It  required  the  subjects  to  identify  the  numbered  parts  on  a detailed  drawing  of  a heart.  Average  Kuder- 
Richardson  Formula  20  Reliability  coefficients  from  a random  sampling  of  prior  studies  for  this  identification  test 
was  .81. 

Terminology  Test 

This  test  of  20  item  multiple  choice  questions  was  designed  to  measure  knowledge  of  specific  facts,  terms,  and 
definitions  in  the  instructional  material.  The  objectives  measured  by  this  type  of  test  could  be  appropriate  to  all 
content  areas  which  an  understanding  of  the  basic  elements  as  a prerequisite  to  the  learning  of  concepts,  rules,  and 
principles.  Average  Kuder-Richardson  Formula  20  Reliability  coefficients  froma  random  sampling  of  prior  studies 
for  this  terminology  test  was  .83. 

Comprehension  Test 

This  test  of  20-item  multiple  choice  questions  was  designed  to  measure  the  students’  ability  to  comprehend  the 
instructional  material  content.  This  comprehension  test  required  the  subjects  to  have  a thorough  understanding  of  the 
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parts  of  the  human  heart,  their  functions,  and  the  internal  processes  that  took  place  during  the  systolic  and  diastolic 
phases.  It  was  designed  to  measure  a type  of  understanding  in  which  the  individuals  could  use  the  information  to 
explain  some  other  phenomenon.  Average  Kuder- Richardson  Formula  20  Reliability  coefficients  from  a random 
sampling  of  prior  studies  for  this  comprehension  test  was  .77. 

Total  Criterion  Test 

The  items  in  the  identification,  terminology  and  comprehension  tests  were  combined  to  provide  a 60  item  total 
criterion  score.  The  score  was  used  to  measure  students’  total  performance  on  the  three  individual  criterion 
measures. 


Design  of  the  study 

The  design  of  the  study  was  1x4  posttest  only.  The  independent  variable  was  concept  mapping  strategy  that 
included  4 levels:  one  control  group  and  three  concept  mapping  strategy  groups  described  as  the  treatments  above. 
The  dependent  variable  was  the  individual  students’  achievement  measured  by  the  identification,  terminology, 
comprehension,  and  total  criterion  tests.  One-way  analysis  of  variance  was  run  to  determine  the  differences  between 
groups.  Alpha  level  was  set  at  the  .05.  Figure  4 shows  the  basic  design  of  this  study. 
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FIGURE  4.  Design  of  the  study 
Treatments 


Tests 


Online  instruction 


Identification 


Online  instruction 


Concept 

Identifying 

Mapping 


Proposation 

- 

r '1 

Identifying 

Proposition 

Identifying 

Mapping 


>*1 


Identification 


j ^^^rminolog^^j  t 


Identification 


H Student  | IT ^ JT 1 

Generated  I Identification  I Terminology  I 

Mapping  J 


Results  and  conclusions 

Table  1 presents  the  number  of  subjects  in  each  treatment,  the  mean  scores  and  standard  deviation  achieved  by 
the  students  in  each  treatment  on  each  criterion  test. 

Table  1:  The  Number  of  Subjects,  Means  Scores,  and  Standard  Deviation  on  Criterion  Tests 

Criterion  Tests 


Identification  Terminology  Comprehension  Total 


Treatment  Group 

n 

M 

SD 

M 

SD 

M 

SD 

M 

SD 

Treatment  1 

29 

7.10 

3.47 

8.38 

3.55 

6.86 

2.64 

22.34 

7.64 

Treatment  2 

27 

11.67 

4.95 

11.85 

4.39 

11.00 

4.79 

34.52 

13.22 

Treatment  3 

24 

10.00 

5.49 

10.25 

4.67 

9.42 

4.45 

29.67 

13.56 

Treatment  4 

25 

11.52 

5.24 

11.68 

4.05 

8.36 

3.39 

31.56 

11.78 

Note:  Maximum  score  possible  = 20.00 

Total  = Identification  + Terminology  + Comprehension 


One-way  analysis  of  variance  (ANOVA)  was  conducted  to  analyze  the  data  collected.  Alpha  level  was  set  at  the 
.05.  The  results  revealed  that  there  were  significant  differences  existed  among  treatment  groups  on  all  criterion  tests 
(Table  2). 
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Table  2:  Results  of  ANOVA 


Sum  of  Squares 

df 

Mean  Square 

F 

Sig 

Identification 

Between  Groups 

376.061 

3 

125.354 

5.450 

.002 

Within  Groups 

2322.930 

101 

22.999 

Total 

2698.990 

104 

Terminology 

Between  Groups 

216.054 

3 

72.018 

4.156 

.008 

Within  Groups 

1750.175 

101 

17.328 

Total 

1966.229 

104 

Comprehension 

Between  Groups 

253.092 

3 

84.364 

5.250 

.002 

Within  Groups 

1623.042 

101 

16.070 

Total 

1875.133 

104 

Total 

Between  Groups 

2268.871 

3 

756.290 

5.561 

.001 

Within  Groups 

13736.786 

101 

136.008 

Total 

16005.657 

104 

Post  hoc  tests  (both  Scheffe  and  Dunnet  C)  were  run  in  SPSS  Program  to  identify  the  specific  differences 
existed  between  treatment  groups.  Table  3 presents  the  results  of  the  post  hoc  tests. 

Table  3:  Results  of  Post  Hoc  Tests  of  Scheffe  and  Dunnett  C 


Dependent 

Variable  (I)  Groups  (J)  Groups 

Mean 
Differe 
nee  (I- 

J) 

Std 

Error 

Sig 

95% 

Confidence 

Intervai 

Lower 

Bound 

Upper 

Bound 

Identification  Scheffe  Control  Concept  Identifying 

Group 

Proposition  Identifying 
Student  generated 
mapping 

-4.56* 

1.28 

.007 

-8.2i 

-.92 

-2.90 

1.32 

.195 

-6.66 

.87 

4.42* 

1.31 

.013 

-8.14 

-.70 

Dunnett  Control  Concept  Identifying 

C Group 

Proposition  Identifying 
Student  generated 
mapping 

4.56* 

1.28 

-7.71 

-1.42 

-2.90 

1.32 

-6.46 

.67 

4.42* 

1.31 

-7.80 

-1.03 

Terminology  Scheffe  Control  Concept  Identifying 

Group 

Proposition  Identifying 
Student  generated 
mapping 

-3.47* 

1.11 

.025 

-6.64 

-.31 

-1.87 

1.15 

.452 

-5.14 

1.40 

-3.30* 

1.16 

.043 

-6.53 

-7.06 

Dunnett  Control  Concept  Identifying 

C Group 

Proposition  Identifying 
Student  generated 
mapping 

-3.47* 

1.11 

-6.41 

-.54 

-1.87 

1.15 

-5.07 

1.32 

-3.30* 

1.14 

-6.17 

-.43 

Comprehens  Scheffe  Control  Concept  Identifying 

ion  Group 

Proposition  Identifying 
Student  generated 
mapping 

4.14* 

1.07 

.003 

-7.19 

-1.09 

-2.25 

1.11 

.156 

-5.70 

.59 

-1.50 

1.09 

.601 

-4.61 

-1.61 

Dunnett  Control  Concept  Identifying 

C Group 

Proposition  Identifying 

4.14* 

1.07 

-7.00 

-1.28 

-2.55 

1.11 

-5.40 

.29 
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Student  generated 
mapping 

-1.50 

1.09 

-4.06 

-1.06 

Total  Tests  Scheffe  Control  Concept  Identifying 

Group 

Proposition  Identifying 

Student  generated 
mapping 

-12.17* 

3.12 

.003 

21.04 

-3.31 

-7.32 

3.22 

.167 

16.47 

1.83 

-9.22* 

3.18 

.044 

18.26 

-.17 

Dunnett  Control  Concept  Identifying 

C Group 

Proposition  Identifying 

Student  generated 
mapping 

-12.17* 

3.12 

20.16 

-4.19 

-7.32 

3.22 

15.90 

1.26 

-9.22* 

3.18 

16.78 

-1.65 

* The  mean  difference  is  significant  at  the  .05  level. 


Both  post  hoc  tests  revealed  that  there  were  significant  differences  between  Treatment  1 (the  control  group)  and 
Treatment  2 (the  concept  identifying  mapping  strategy  group)  on  all  the  criterion  tests;  there  were  significant 
differences  between  Treatment  1 (the  control  group)  and  Treatment  4 (the  student  generated  mapping  strategy 
group)  in  identification,  terminology,  and  total  tests.  Insignificant  differences  were  found  to  exist  among  the  other 
comparisons. 

A number  of  factors  may  have  contributed  to  these  results.  The  significant  differences  that  were  found  to  exist 
between  Treatment  1 (the  control  group)  and  Treatment  2 (the  concept  identifying  mapping  strategy  group)  on  all 
criterion  tests  indicated  that  concept  identifying  mapping  strategy  was  effective  in  facilitating  learning  the  concepts 
and  comprehending  the  instructional  content.  By  using  concept  identifying  mapping  strategy,  the  students’  attention 
was  primarily  focused  on  interacting  with  those  key  concepts  of  instructional  material.  This  meaningful  and  active 
interaction  with  the  concept  maps  not  only  helped  the  students  remember  those  key  concepts  but  also  to  understand 
other  dimension  of  the  content  (other  than  concepts)  in  instructional  material  since  concepts  are  foundations  based 
on  which  comprehension  can  be  built.  This  also  explains  that  those  students  who  used  concept  identifying  mapping 
strategy  did  well  not  only  on  the  identification  and  terminology  tests  but  also  on  the  comprehension  test. 

The  significant  differences  existed  between  Treatment  1 (the  control  group)  and  Treatment  4 (the  student 
generated  mapping  strategy  group)  on  the  identification,  terminology,  and  total  tests.  This  finding  indicated  that 
student  generated  concept  mapping  strategy  was  effective  in  facilitating  conceptual  learning.  The  pre-requisite  to 
construct  a meaningful  concept  map  was  to  identify  those  key  concepts  and  propositions  among  them.  The  student 
generated  concept  mapping  strategy  helped  the  students  interact  with  the  instructional  material  by  identifying  the 
key  concepts,  the  propositions,  and  the  structure  of  the  instructional  materials.  Consequently  it  resulted  in 
significantly  higher  achievement  on  the  identification,  terminology  and  total  criterion  tests.  However,  the  student 
generated  concept  mapping  strategy  did  not  significantly  effect  the  achievement  measured  by  comprehension  test. 
One  explanation  for  this  finding  may  be  that  the  student  generated  concept  mapping  was  a very  demanding  cognitive 
activity  that  required  relatively  long  time  and  sustained  efforts.  The  analysis  of  the  concept  maps  generated  by  the 
students  and  the  observation  made  during  the  implementation  of  the  study  revealed  that  most  of  the  concept  maps 
were  only  of  identifying  and  representing  key  concepts  of  the  instructional  material.  They  failed  to  map  out  the 
complex  relationships  among  those  key  concepts  identified,  which  would  instigate  deeper  processing  and 
comprehending  of  the  instructional  material.  Wachter  (1993)  suggested  that  comprehension  tended  to  be  enhanced 
when  the  students  constructed  their  own  associations  and  elaborations  during  reading  tasks.  It  was  also  observed  that 
during  the  study,  a number  of  students  were  having  difficulties  in  constructing  their  concept  maps.  This  also 
distracted  their  efforts  from  active  processing  of  the  information  needed  to  comprehend  the  instructional  material  in 
depth  during  the  study. 

There  were  insignificant  differences  in  achievement  between  Treatment  1 (the  control  group)  and  treatment  3 
(proposition  identifying  mapping  strategy  group)  in  all  criterion  measures.  This  means  that  Treatment  3 (the 
proposition  identifying  mapping  strategy)  failed  to  significantly  effect  the  learning.  The  concept  mapping  activities 
in  this  group  appeared  too  complicated  for  the  students  to  complete  within  their  attention  span.  The  analysis  of  the 
concept  maps  completed  by  the  students  and  the  observation  made  during  the  implementation  of  the  study  revealed 
over  one-third  of  the  students  in  this  group  did  not  complete  the  required  concept  mapping  activities.  As  a result,  it 
was  not  effective  in  eliciting  the  proper  level  of  cognitive  process  it  had  meant  to  instigate.  Consequently,  the 
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intensity  and  quality  of  the  interactions  of  the  students  with  the  instructional  material  were  degraded  and  no 
significant  results  were  found  on  any  of  the  criterion  tests. 

The  above  explanation  can  also  be  used  to  explain  the  results  of  no  significant  differences  among  Treatment  2 
(the  concept  identifying  mapping  strategy  group),  Treatment  3,  (the  proposition  identifying  mapping  strategy 
group),  and  Treatment  4 (the  student  generated  mapping  strategy  group)  in  all  criterion  measures.  Additionally,  the 
concept  mapping  activities  in  all  treatments  dealt  with  learning  the  instructional  material  in  general.  They  did  not 
aim  at  helping  students  to  comprehend  particular  difficult  areas  of  the  instructional  material,  which  had  greater 
potential  for  improvement.  As  a result,  those  difficult  areas  of  the  instructional  material  remained  untouched  in  this 
study 

Implications 

This  study  attempted  to  examine  the  effects  of  the  three  concept  mapping  strategies  in  facilitating  learning 
achievement.  The  alpha  level  was  set  at  .05.  There  were  statistically  significant  differences  found  between  the 
control  group  and  the  concept  identifying  mapping  strategy  group  in  all  criterion  tests.  The  significant  differences 
were  also  found  between  the  control  group  and  the  student  generated  mapping  strategy  group  in  the  identification, 
terminology  and  total  criterion  tests.  There  were  no  statistically  significances  found  among  the  three  concept 
mapping  strategy  groups.  In  the  future  study,  it  is  recommended  that  in  studying  the  different  mapping  strategies,  the 
workload  of  the  concept  mapping  activities  in  each  treatment  be  carefully  considered.  Too  much  workload  in  the 
treatment  would  not  likely  produce  the  intended  results.  Roshan  and  Dwyer  (1998)  maintained  that  when  the  human 
information  processing  systems  were  under  the  stress  of  processing  overloading,  performance  would  become 
degraded.  Additionally,  the  concept  mapping  strategies  should  aim  at  specific  difficult  areas  of  learning  that  have 
high  potentials  for  improvement.  The  effective  concept  mapping  activities  should  help  overcome  those  difficulties 
the  students  have  in  processing,  organizing  and  applying  the  information.  It  is  highly  possible,  if  concept  mapping 
activities  had  been  properly  devised,  there  might  have  been  the  statistically  significance  found  among  those 
treatment  groups. 
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Impact  of  Vicarious  Learning  Experiences  and  Goal  Setting  on  Preservice 
Teachers’  Self-Efficacy  for  Technology  Integration:  A Pilot  Study 

Ling  Wang 
Purdue  University 


Abstract 

This  pilot  study  was  designed  to  explore  how  vicarious  learning  experiences  and  goal  setting  influence 
preservice  teachers’  self-efficacy  for  integrating  technology  into  the  classroom.  In  this  pilot  study,  twenty  students 
from  the  Teacher  Education  program  at  a Mid-western  university  participated  and  were  assigned  into  four 
experimental  conditions.  Results  showed  the  individual  and  collective  effects  of  vicarious  experiences  and  goal 
setting  on  the  participants’  judgments  of  self-efficacy  for  technology  integration.  More  data  were  simulated  based  on 
the  result  of  this  pilot  study  to  detect  the  effect  of  vicarious  learning  experiences  and  goal  setting  when  the  sample 
size  was  80  participants. 

Theoretical  Framework 

Despite  the  increased  availability  and  support  for  computers  (Zehr,  1997,  1998),  relatively  few  teachers 
have  fully  integrated  computers  into  their  classroom  teaching  (Marcinkiewicz,  1996).  There  is  substantial  evidence 
to  suggest  that  teachers'  beliefs  in  their  capacity  to  work  effectively  with  technology,  that  is,  their  computer  self- 
efficacy.  is  a significant  factor  in  determining  patterns  of  classroom  computer  use  (Oliver  & Shapiro,  1993).  The 
studies  on  teachers'  self-efficacy  beliefs  provide  sufficient  reason  to  undertake  further  investigations  in  this  area  and 
to  consider  approaches  to  teacher  education  and  professional  development  that  might  be  effective  in  increasing  self- 
efficacy  for  teaching  with  technology. 

Bandura  (1986)  identified  four  sources  of  information  used  to  judge  self-efficacy:  successful  performance 
attainment;  observing  the  performances  of  others  (vicarious  learning):  verbal  persuasion  indicating  that  one 
possesses  certain  capabilities;  and  physiological  states  by  which  one  judges  capability,  strength,  and  vulnerability. 
Although  performance  accomplishments  are  considered  to  be  the  most  robust  source  of  self-efficacy  information, 
vicarious  learning  also  provides  a powerful  source  (Bandura,  1986,  1997).  That  is,  viewing  others  successfully 
accomplish  a particular  task  can  increase  one’s  perceptions  of  others’  efficacy  as  well  as  one’s  own  self-efficacy  for 
similar  tasks  (Bandura,  1997).  The  effects  of  vicarious  learning  merit  further  examination. 

While  novice  learners  can  acquire  skills  and  strategies  from  social  modeling,  when  performing 
independently  they  are  likely  to  over-  or  underestimate  their  capabilities  (Schunk,  2001).  However,  by  making 
learning  goals  explicit,  students’  judgments  of  progress,  as  well  as  their  judgments  of  self-efficacy,  increase  in 
accuracy  and  strength  (Schunk,  2001).  Goals  can  motivate  behavior  and  inform  people  about  their  capabilities 
(Bandura,  1997;  Schunk,  1996).  When  students  are  given  or  establish  a goal,  they  may  experience  a sense  of 
efficacy  for  attaining  it. 

Based  on  the  literature  described  above,  this  study  examined  the  impact  of  vicarious  learning  experiences 
and  goal  setting  on  preservice  teachers’  self-efficacy  for  technology  integration. 

Purpose  of  the  Study 

This  study  explored  how  vicarious  learning  experiences  and  goal  setting  influence  preservice  teachers’  self- 
efficacy  for  integrating  technology  into  the  classroom.  Specifically,  this  study  was  guided  by  the  following  research 
question: 

What  are  the  individual  and  collective  effects  of  vicarious  experiences  and  goal  setting  on  preservice 
teachers’  judgments  of  self-efficacy  for  technology  integration? 

Based  on  the  self-efficacy  literature  described  above,  it  was  hypothesized  that  preservice  teachers  who 
engaged  in  vicarious  experiences,  related  to  successful  technology  integration,  would  experience  a significantly 
greater  increase  in  judgments  of  computer  self-efficacy  than  those  who  did  not  engage  in  these  vicarious 
experiences.  Furthermore,  it  was  hypothesized  that  preservice  teachers  who  engaged  in  goal  setting,  related  to 
increasing  their  technology  integration  skills,  would  experience  a significantly  greater  increase  in  judgments  of 
computer  self-efficacy  than  those  who  did  not  engage  in  goal  setting.  Finally,  it  was  hypothesized  that  preservice 
teachers  who  engaged  in  vicarious  experiences  and  goal  setting  would  demonstrate  the  greatest  increases  in 
judgments  of  computer  self-efficacy  compared  to  students  who  engaged  in  either  one  of  these  conditions  alone. 
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Methodology 


Sample 

Participants  were  solicited  from  a four- week  three-credit  educational  technology  course  at  a large  Mid- 
western university.  They  were  sophomores  in  the  Teacher  Education  program  and  their  age  range  was  from  18  to 
25. Thirty-one  students  signed  the  informed  consent  form  and  agreed  to  participate,  but  only  twenty  of  them  finally 
showed  up  in  the  study.  The  thirty-one  participants  were  randomly  assigned  into  four  experimental  conditions.  The 
numbers  of  participants  who  finally  showed  up  for  the  four  experimental  conditions  were  6,  6,  4,  and  4 (Note:  The 
unequal  group  size  was  due  to  the  dropouts  from  the  study). 

Research  Design 

A 2 X 2 (Vicarious  Experiences:  use  of  VisionQuest  software  [Note:  VisionQuest  is  an  instructional  tool 
designed  to  provide  opportunities  for  users  to  explore  models  of  effective  technology  integration]  vs.  no  use  of 
VisionQuest  software  x Presence  of  Goal  Setting:  goal  setting  vs.  no  goal  setting)  mixed  factorial  research  design 
was  used  to  examine  how  vicarious  experiences  and  learning  goals  individually,  as  well  as  collectively,  impacted 
preservice  teachers’  judgments  of  their  computer  self-efficacy.  These  independent  variables  were  combined  to  form 
four  experimental  conditions:  (a)  NVQ  / NG:  no  use  of  VisionQuest  software  and  no  learning  goals  (also  defined  as 
control  group),  (b)  NVQ  / G:  no  use  of  VisionQuest  software  but  with  learning  goals,  (c)  VQ  / NG:  use  of 
VisionQuest  software  but  without  learning  goals,  and  (d)  VQ  / G:  use  of  VisionQuest  software  and  with  learning 
goals. 

Data  Collection  and  Analysis  Strategies 

Demographic  data,  including  information  about  gender,  class,  and  previous  computer  experiences  were 
collected  from  all  participating  students.  A Likert -style  survey  measuring  participants’  self-efficacy  served  as  the 
primary  data  source  and  was  administered  at  the  end  of  each  experimental  condition.  The  survey  was  constructed  by 
the  researcher  after  consulting  with  content  area  experts.  The  survey  included  21  items  regarding  participants' 
confidence  for  technology  use  (see  the  attachment).  The  participants  were  asked  to  rate  their  level  of  agreement 
(from  1 -strongly  disagree  to  5-strongly  agree)  with  statements  related  to  their  possession  of  confidence  regarding 
technology  use  (e.g.,  “I  feel  confident  that  I understand  computer  capabilities  well  enough  to  maximize  them  in  my 
classroom.”  “I  feel  confident  I can  regularly  incorporate  technology  into  my  lessons,  when  appropriate  to  student 
learning.”).  Cronbach  alpha  was  calculated  to  determine  the  reliability  of  the  survey  items.  The  alpha  coefficient  was 
0.9293. 

Eor  the  groups  that  engaged  in  vicarious  experiences,  participants  explored  the  VisionQuest  CD-ROM  and 
observed  the  exemplar  teachers’  technology  use  and  classroom  management  strategies.  Eor  the  groups  that  engaged 
in  learning  goals,  participants  were  given,  at  the  beginning  of  the  experiment,  3 to  5 specific  goals  related  to  learning 
about  technology  integration. 

Means  and  standard  deviations  for  each  experimental  condition  were  calculated.  Two-way  ANOVA  was 
used  to  determine  the  main  effects  of  IVj  (vicarious  learning  experiences  through  viewing  VisionQuest  CD-ROM) 
and  IV2  (learning  goals)  and  the  interaction  of  IVi  and  IV2.  Qne-way  ANOVA  was  then  used  to  determine  which  of 
the  four  experimental  conditions  was  significantly  different  from  which. 

Results 

The  four  experimental  conditions  had  the  following  means  and  standard  deviations:  VQ  / G (mean=  4.55, 
SD=0.33);  VQ  / NG  (mean=4.30,  SD=0.41);  NVQ  / G (mean=4.24,  SD=0.52);  and  NVQ  / NG  (mean=  3.57, 
SD=0.14). 

The  E value  (F=5.57,  p=0.0082,  df=3,  16)  from  the  two-way  ANOVA  indicated  that  there  was  at  least  one 
main  effect  of  the  two  factors  was  significant.  Then  when  examining  the  F values  for  each  of  the  two  factors  and  the 
interaction  of  the  two  factors,  the  researcher  found  that  the  vicarious  learning  factor  had  a significant  main  effect  on 
the  self-efficacy  (F=9.18,  p=0.0080,  df=l,  16),  the  learning  goals  factor  also  had  a significant  main  effect  on  the 
self-efficacy  (F=6.06,  p=0.0256,  df=l,  16),  but  the  interaction  of  the  two  factors  was  not  significant  (F=1.48, 
p=0.2407,  df=l,  16).  The  following  graph  helped  to  illustrate  the  main  effects  and  the  interaction: 

Graph  1 . Main  Effects  and  Interaction  Effect 
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The  F value  from  the  (F=5.57,  p=0.0082,  df=3,  16)  from  the  one-way  ANOVA  indicated  that  the  variable — different 
conditions  of  the  experiment — had  a significant  effect  on  the  self-efficacy.  When  examining  which  conditions  were 
significantly  different  from  which  conditions,  the  researcher  found  that  the  following  between-condition  differences 
were  significant  at  the  0.05  level:  VQ  / G - NVQ  / NG  (the  mean  difference=0.98),  and  VQ  / NG  - NVQ  / NG  (the 
mean  difference=0.73).  The  following  graph  helped  to  illustrate  the  difference  between  the  four  experimental 
conditions: 

Graph  2.  Difference  between  Experimental  Conditions 
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Because  this  was  a pilot  study  with  only  twenty  participants,  the  small  sample  size  of  the  study  could  be  a 
factor  affecting  the  results  (or  at  least  part  of  the  results).  Therefore,  the  researcher  simulated  another  data  set  based 
on  the  means  and  standard  deviations  of  the  four  experimental  conditions  of  the  pilot  study.  After  simulation,  the 
sample  size  was  enlarged  to  80  participants  with  20  participants  in  each  of  the  four  conditions. 

The  F value  (F=27.5,  p<0.0001 , df=3,  76)  from  the  two-way  ANOVA  indicated  that  there  was  at  least  one 
main  effect  of  the  two  factors  was  significant.  Then  when  examining  the  F values  for  each  of  the  two  factors  and  the 
interaction  of  the  two  factors,  the  researcher  found  that  the  vicarious  learning  factor  had  a significant  main  effect  on 
the  self-efficacy  (F=40.46,  p<0.0001,  df=l,  76),  the  learning  goals  factor  also  had  a significant  main  effect  on  the 
self-efficacy  (F=38.83,  p<0.0001,  df=l,  76),  but  the  interaction  of  the  two  factors  was  not  significant  (F=3.21, 
p=0.0773,df=l,76). 
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The  F value  from  the  (F=27.5,  p<0.0001,  df=3,  76)  from  the  one-way  ANOVA  indicated  that  the 
variable — different  conditions  of  the  experiment — ^had  a significant  effect  on  the  self-efficacy.  When  examining 
which  conditions  were  significantly  different  from  which  conditions,  the  researcher  found  that  the  following 
between-condition  differences  were  significant  at  the  0.05  level:  VQ  / G - VQ  / NG,  VQ  / G - NVQ  / G,  VQ  / NG  - 
NVQ  / NG,  NVQ  / G - NVQ  / NG,  and  VQ  / NG  - NVQ  / NG. 

Conclusion 

This  study  was  the  pilot  study  for  the  researcher’s  dissertation  research.  From  the  results  of  the  data 
analysis,  it  can  be  concluded  that  the  preservice  teachers  who  engaged  in  vicarious  experiences,  related  to  successful 
technology  integration,  experienced  a significantly  greater  increase  in  judgments  of  computer  self-efficacy  than 
those  who  did  not  engage  in  these  vicarious  experiences.  Also,  the  preservice  teachers  who  engaged  in  learning 
goals,  related  to  increasing  their  technology  integration  skills,  experienced  a significantly  greater  increase  in 
judgments  of  computer  self-efficacy  than  those  who  did  not  engage  in  learning  goals.  In  addition,  the  preservice 
teachers  who  engaged  in  vicarious  experiences  (with  or  without  learning  goals)  demonstrated  significant  increases  in 
judgments  of  computer  self-efficacy  compared  to  those  who  engaged  in  neither  of  the  two  conditions.  However,  the 
preservice  teachers  who  engaged  in  learning  goals  alone  did  not  demonstrate  significant  increases  in  judgments  of 
computer  self-efficacy  compared  to  those  who  engaged  in  neither  of  the  two  conditions.  Similarly,  the  preservice 
teachers  who  engaged  in  both  vicarious  experiences  and  learning  goals  did  not  necessarily  demonstrate  significant 
increases  in  judgments  of  computer  self-efficacy  compared  to  those  who  engaged  in  either  of  the  two  conditions 
alone. 

With  the  findings  from  the  simulated  data  set,  the  researcher  concluded  that  the  preservice  teachers  who 
engaged  in  both  vicarious  experiences  and  learning  goals  did  demonstrate  significant  increases  in  judgments  of 
computer  self-efficacy  compared  to  those  who  engaged  in  either  of  the  two  conditions  alone,  in  addition  to  the 
significant  increases  detected  in  preservice  teachers’  judgments  of  computer  self-efficacy  when  only  one  condition 
was  present. 

Educational  Implication 

The  results  of  this  study  will  contribute  to  the  existing  body  of  literature  in  two  significant  ways:  (1)  by 
describing  how  preservice  teachers  benefit  from  observing  teacher  models  presented  via  multimedia  case  examples, 
such  as  those  featured  on  VisionQuest,  and  (2)  by  describing  how  preservice  teachers  benefit  from  learning  goals  for 
technology  integration.  From  an  instructor’s  perspective,  the  use  of  electronic  models  via  multimedia  and  the 
incorporation  of  learning  goals  can  positively  impact  the  authentic  nature  of  a course  and  simultaneously  increase 
the  confidence  and  integration  beliefs  of  students.  This  type  of  modeling  and  goal  setting  can  help  preservice 
teachers  develop  a vision  for  what  technology  integration  looks  like  in  real  classrooms  as  well  as  strategies  for 
implementing  those  visions. 
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Abstract 

Through  reconceptualization  of  the  Clark  and  Kozma  debate,  this  paper  attempts  to  foster  an  awareness  of 
the  need  for  constructive  criticism  and  define  a discipline  for  an  intellectual  dialogue  in  the  field  of  Instructional 
Technology.  Analyzing  the  differences  between  debate  and  dialogue,  the  authors  show  how  Clark  and  Kozma’ s 
initial  debate  has  grown  into  a start  of  a dialogue.  Popper’s  concept  of  falsification  and  refutation  is  one  of  the 
possible  guidelines  for  starting  the  process  of  dialogue. 

Future  developments  of  this  paper  can  continue  to  refine  the  process  of  dialogue  to  provide  a mean  for 
improving  the  discipline  of  the  dialogue  for  the  purpose  of  informing  the  IT  field. 

Introduction 

While  theories  in  the  field  of  Instructional  Technology  (IT)  are  proliferating  at  an  alarming  rate,  there  has 
been  little  history  of  criticism  or  framework  of  analysis  for  discussing  competing  perspectives  for  the  purpose  of 
enriching  the  field.  As  Steiner  (1988,  p.  xi)  points  out  “theory  is  built  upon  existing  theory.  Consequently,  to  build 
theory  one  must  be  able  to  criticize  theory.  One  must  be  able  to  achieve  an  understanding  of  extant  theory  and  to 
judge  what  needs  to  be  done,  if  anything,  to  the  theory.  Only  then  is  one  in  position  to  make  constructive  moves.’’ 
One  such  constructive  move  would  be  to  engage  in  intellectual  dialogues,  allowing  for  a discussion  of  similarities 
and  differences  which  can  be  brought  into  focus  in  order  to  enrich  the  field  (Schwen,  2001).  A lack  of  dialogue 
poses  a problem  both  for  practitioners  as  well  as  researchers  in  IT.  W ithout  explicitly  stated  constraints  or  valid 
frameworks  of  analysis,  the  applied  usefulness  of  theories  is  limited. 

The  goals  of  this  paper  are  to  foster  an  awareness  of  the  need  for  constructive  criticism,  capture  any  current 
dialogue  in  the  field  and  define  a discipline  for  dialogue.  Instances  of  this  dialogue  will  be  illustrated  and  analyzed 
and  a set  of  guidelines  will  be  provided  that  can  foster  constructive  dialogue  in  the  field. 

Definition  of  the  terms 

While  we  agree  that  “theory’’  and  “practice’’  are  not  dichotomous,  if  the  two  concepts  were  presented  as  a 
continuum,  with  pure  theory  at  one  end  and  absolute  practice  at  the  other,  somewhere  near  the  practice  end  of  it, 
there  would  be  a gap  into  which  falls  any  relationship  that  could  be  established  between  theoreticians  and 
practitioners.  It  is  a gap  that  must  be  bridged  if  theories  are  to  have  any  useful  application.  “We  must  not  negate 
practice  for  the  sake  of  theory.  To  do  so  would  reduce  theory  to  a pure  verbalism  or  intellectualism.  By  the  same 
token,  to  negate  theory  for  the  sake  of  practice. . .is  to  run  the  risk  of  losing  oneself  in  the  disconnectedness  of 
practice’’  (Freire  and  Macedo,  1995,  p.382).  Those  authors  advocate  a unity  between  theory  and  practice. 

Schon’s  (1983)  concept  of  reflective  practice  is  an  attempt  to  bring  both  together.  He  points  out  that 
knowledge  is  implicit  in  any  competent  professional.  Making  judgments  and  decisions  are  any  professional  practice. 
However,  professionals  have  some  difficulty  to  state  the  rules  or  theories  upon  which  are  based  (Schon,  1983). 

Schon  (1987)  calls  this  practical  knowledge  as  “knowing-in-action”.  Professionals  come  to  know  in  action  through  a 
process  of  reflection  - “reflection-in-action’’.  Ambiguity,  uncertainty,  uniqueness,  and  values  conflict  characterize 
practice.  It  involves  thoughtfully  consideration  of  one’s  own  experiences  in  applying  knowledge  to  practice  and 
transforming  indetermination  into  something  that  he  or  she  knows  how  to  deal  with. 

A possibility  of  fostering  environments  for  dialogue  can  be  found  in  Karl  Popper’s  concept  of  falsification. 
Popper  proposes  a new  kind  of  critical  method  in  opposition  to  inductivism,  arguing  against  the  inductivists’  search 
for  positive  instances.  Popper  emphasizes  that  a scientific  theory  can  never  be  “verified’’  but  can  only  be  “falsified”. 
We  interpret  his  concept  of  falsification  as  “looking  for  the  data  that  reveals  the  instances  where  the  theory  doesn’t 
work,”  and  this  has  resonance  of  Cronbach  (1980,  p.  103)  who  said,  “ The  job  of  validation  is  not  to  support  an 
interpretation,  but  to  find  out  what  might  be  wrong  with  it”.  The  aim  of  researches  should  be  “regulative  ideal”  of 
seeking  knowledge  (Phillips,  p.xx).  The  fact  that  we  are  fallible  does  not  undermine  the  validity  of  the  idea,  Phillips 
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emphasizes,  “because  even  if  we  fail  to  find  an  answer  or  find  the  answer  we  have  accepted  in  the  past  is  mistaken, 
is  itself  an  advance  in  knowledge.” 

In  contrast  to  the  traditional  definition  of  theory,  which  utilizes  a verification  model.  Popper  points  out  that 
theories  are  not  as  firm  and  strong  as  we  once  supposed.  He  strongly  believes  that  it  is  easy  to  obtain  confirmations 
or  verifications,  for  nearly  every  theory — if  we  look  for  confirmations  (Popper,  1962).  By  “testing”,  he  means 
testing  for  refutation.  If  the  theory  survives  rigorous  testing,  then  it  can  be  said  to  have  acquired  corroborating 
evidence,  meaning  that  the  theory  does  not  conflict  with  the  available  experiential  data.  The  use  of  the  term 
“corroboration”  in  the  place  of  “probability”  further  shows  Popper’s  distance  from  the  logical  positivists.  He  also 
argues  that  a distinctively  scientific  approach  requires  the  researcher  to  look  for  weaknesses,  drawbacks  or 
inconsistency  with  empirical  data.  His  concept  of  falsification  refers  to  the  procedure  that  effectively  refutes  a 
theory.  In  order  to  test  a theory  through  the  procedure  of  falsification,  the  theory  has  to  assure  potential 
falsifiability,  are  requirements  that  guarantees  the  empirical  character  of  a system  of  statements  (Corvi,  1996). 

Popper  uses  the  metaphor  of  science  as  a ‘searchlight’  in  explaining  the  role  of  science.  This  image  is  in 
sharp  contrast  with  the  classical  theory  of  science  as  an  accumulation  of  observations;  it  emphasizes  the  theoretical 
element  as  the  guide  and  criterion  of  observation  (Corvi,  1996).  In  this  metaphor.  Popper  points  out  the  dual  role  of 
science:  (1)  solving  the  existing  problem  while  creating  new  ones,  and  (2)  using  observation  as  well  as  encouraging 
others  to  consider  different  ways  of  interpreting  observations  already  made  (Popper,  1962).  In  Popper’s  argument, 
the  scientist’s  goal  is  not  to  accept  the  observation  of  the  empirical  data  as  a priori  but  to  posit  them  as  tentative 
conjecture  that  can  anytime  be  refuted. 

The  “sophisticated  methodological  falsification”  presented  by  Lakatos  takes  Popper’s  concept  still  further 
(Lavor,  1998).  In  this  modification,  the  concept  of  “research  programs”  and  the  methodology  of  scientific  research 
programs  can  be  interpreted  as  Lakatos’  attempt  to  answer  Popper’s  question  of  demarcation.  A scientific  program  is 
the  cluster  of  ideas  and  discussions,  and  is  comprised  of  the  “hard  core”,  ideas  that  cannot  be  falsified  with  a single 
instance  and  its  “protective  belt,”  the  shield  against  which  the  attempts  of  refutation  are  being  bounced.  A program 
is  said  to  be  theoretically  progressive  if  modifications  lead  to  unexpected  predictions  and  empirically  progressive  if 
some  of  these  predictions  are  corroborated  (Larvor,  1998).  An  instance  of  falsified  theory  will  still  lead  to  the 
progress  of  the  programs  through  “the  regulative  ideal.” 

The  5 A’s  of  debate  and  dialogue 

In  order  to  inform  the  reader  of  our  frame  of  reference  with  regard  to  debate  and  dialogue,  we  offer  the 
following  definitions.  Debate,  according  to  Webster’s  Dictionary  (1982),  is  “to  fight  or  contend,  to  dispute  about  or 
to  argue”.  It  carries  with  it  overtones  of  controversy  and  strife.  In  contrast  to  this,  some  authors  (Boone,  2001;  Roth, 
1994;  Stewart,  n.d.),  define  dialogue  as  the  interchange  of  ideas  that  seeks  to  establish  greater  learning  or 
understanding  in  the  context  of  mutual  harmony.  However,  in  the  literature  “dialogue”  is  often  related  to  concepts 
such  as  “conversation”,  “interaction”,  “communication”  and  “bridging”.  The  5 A’s  of  debate  or  dialogue  table 
(Table  1)  offers  a convenient  way  to  differentiate  the  two. 
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Debate 

Dialogue 

Assumptions 

Assuming  there  is  only  one  right  answer 
(and  you  have  it). 

Assuming  that  other  people  have  pieces  of 
the  answer. 

Anticinatorv 

Advance  communication  is  minimal  and 
not  specifically  linked  to  what  follows. 

Advance  communication  is  essential  in 
order  to  prepare  participants  for  the 
process. 

Audience 

Participants  speak  as  representatives  of 
the  group; 

• They  are  known  for  propounding  a 
carefully  crafted  position;  and 

• Their  behavior  conforms  to 
stereotypes. 

Participants  speak  as  individuals  from 
their  unique  experiences; 

• They  are  not  outspoken  leaders,  but 
have  a range  of  different  experiences; 

• Their  behavior  will  vary  from 
stereotypic  images. 

Atmosphere 

Combative: 

• Attempting  to  prove  the  other  side 
wrong,  thus  declaring  you  the 
“winner.” 

• Defending  your  views  and 
assumptions 

• Determination  to  be  right. 

Hearing  (as  opposed  to  listening) 

• To  find  flaws  and  make  counter- 
arguments; 

• To  criticize  the  other  side’s  point  of 
view. 

(Belittles  others  and  is  divisive) 

Collaborative: 

• Finding  common  understanding  and 
common  ground. 

• Bringing  your  assumptions  into  the 
open  for  inspection  and  discussion; 

• Admitting  that  other’ s thinking  may 
improve  your  own. 

Listening 

• For  understanding  and  finding  a 
basis  for  agreement; 

• Being  willing  to  examine  all  points 
of  view;  and 

• Searching  for  strength  and  value  in 
other  positions 

(Involves  concern  for  others.) 

Afterward 

Seeking  an  outcome  that  agrees  with  your 
own,  which  usually  offers  little  new 
information  and  is  predictable. 

Discovering  new  possibilities  and 
opportunities  where  new  information  can 
surface.  Dialogue  liberates  creativity. 

Table  1 - The  5 A’s  of  debate  and  dialogue. 


The  underlying  assumptions  in  debate  and  dialogue  differ  dramatically.  In  debate,  the  assumption  is  that 
there  is  only  one  right  answer  and  that  is  the  one  that  we  possess.  Others  cannot  have  a “right”  answer  unless  it  is  the 
same  as  ours.  The  inverse  of  that  assumption  is  that  if  my  answer  is  “right”,  then  your  must  be  “wrong.”  On  the 
other  hands,  in  a dialogue,  it  is  assumed  that  many  people  may  have  parts  of  the  answer  and  the  goal  is  to  bring  all 
of  the  diverse  pieces  together  to  form  one  new  whole.  Diversity  is  celebrated  and  sought  for  as  a means  of  creating 
something  better  and  more  robust  than  initially  imagined. 

As  a dialogue  is  anticipated,  communication  is  essential  in  order  for  participants  to  be  prepared. 
Communication  sets  the  stage  for  a good  dialogic  process  and  enhances  what  follows.  In  a debate,  advance 
communication  is  often  minimal  and  lacks  congruence  with  the  debate  that  is  to  follow.  Participants  often  contribute 
with  their  own  agendas  in  mind,  with  no  intention  of  coming  to  consensus,  but  they  ultimately  seek  to  persuade 
people  to  “join”  their  viewpoint.  Dialogue  expands  the  number  of  people  committed  to  the  process. 

Participants  in  a debate  speak  as  representatives  of  a “group,”  whose  position  is  carefully  crafted  and  whose 
behavior  conforms  to  stereotypes.  Questions  are  used  as  weapons  for  stabbing  opponents,  not  as  tools  for 
questioning.  In  dialogue,  “the  goal  is  changed  from  conquering  to  growing,  from  silence  to  knowing,  from  telling  to 
asking.  Questions  are  used  as  tools  for  probing,  not  weapons  for  stabbing,”  (The  Conflict  Resolution  Network,  n.d.). 
Those  involved  in  a dialogue  speak  as  unique  individuals,  who  have  a range  of  different  experiences  upon  which  to 
draw  and  share.  There  is  no  outspoken  leader,  but  contributors  on  a level  playing  field. 

The  atmosphere  in  a debate  is  combative,  with  each  side  attempting  to  prove  the  other  side  wrong.  There 
must  be  a winner  and  a loser.  “When  we  argue  or  debate,  we  actually  seek  to  block  the  other’s  contribution  and  limit 
potential  solutions  of  suggestions,  thus  limiting  world  consciousness,”  (Stewart,  n.d.).  Participants  do  not  listen,  the 
only  hear,  with  the  intent  to  find  flaws,  make  counter-arguments  and  criticize.  In  dialogue,  listening  and 
collaborating  are  all  important.  Finding  common  ground  and  being  willing  to  bring  assumptions  into  the  open  for 
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inspection  and  discussion  are  hallmarks  of  an  open  dialogue.  In  dialogue,  participants  search  for  strength  and  value 
in  each  other’s  positions. 

Finally,  a debate  seeks  agreement  with  the  leader’s  position.  The  outcome  is  usually  very  predictable  and 
offers  little,  if  any,  new  information  on  the  topic.  “In  debate,  the  atmosphere  is  usually  threatening,  with 
interruptions  expected.  The  participants  express  unwavering  commitment  to  the  won  point  of  view.  There  is  often  a 
great  deal  of  heat,  but  little  light”  (Stewart,  n.d.).  In  dialogue,  the  goal  is  to  discover  new  possibilities  and 
opportunities  where  information  can  surface.  Dialogue  seeks  to  create  creativity  in  a manner  that  champions  good 
will  and  harmony.  Dialogue  does  not  persuade,  it  invites. 

The  focus  of  dialogue  is  not  to  bring  everyone  to  consensus  on  one  single  strongest  theory  or  practice.  It  is 
an  attempt  to  further  develop,  as  a growing  field,  the  ability  to  think  critically  as  well  as  more  flexibly  through  rich 
discussions  of  various,  competing  viewpoints.  Too  often,  theoretical  debates  resemble  a “dialogue  of  the  deaf’ 
(Fierke  & Nicholson,  2001,  p.9).  In  the  late  Paulo  Freire’s  Pedagogy  of  the  Oppressed,  he  argues  that  “dialogue 
cannot  be  reduced  to  the  act  of  one  person’s  ‘depositing’  ideas  in  another,  nor  can  it  become  a simple  exchange  of 
ideas  to  be  ‘consumed’  by  the  discussants.  Nor  yet  is  it  a hostile,  polemical  argument  between  those  who  are 
committed  neither  to  the  naming  of  the  world,  nor  to  the  search  for  truth,  but  rather  to  the  imposition  of  their  own 
truth. . .It  is  an  act  of  creation;  it  must  not  serve  as  a crafty  instrument  for  the  domination  of  one  person  by  another 
(Freire,  2000,  p.89).  We  argue  that  there  is  not  only  a need  for  dialogue  but  also  for  a framework  for  that  dialogue, 
which  will  not  block  communication  or  divide  perspectives,  but  will  create  synergy  and  dynamic  elucidation  of 
issues  in  the  IT  field.  In  order  to  provide  and  example,  we  examine  one  of  the  most  famous  discussions  in  the  field, 
the  Clark  and  Kozma  debate. 

The  Clark  and  Kozma  debate 

At  first  glance,  an  examination  of  some  of  the  Clark  and  Kozma  arguments  reveal  structures  that  more 
closely  resemble  debate  than  dialogue. 

In  1983  Richard  Clark  wrote  an  article  in  the  Review  of  Educational  Research  -“Reconsidering  Research 
on  Learning  from  Media”  - that  has  started  what  is  known  in  the  field  as  “The  Media  Debate”  or  “Do  Media  Alone 
Influence  Learning  Effectiveness?”  After  reviewing  research  studies  from  1912  to  the  early  1980s,  Clark  claims  that 
media  are  “mere  vehicles  that  deliver  instruction  but  do  not  influence  student  achievement  any  more  than  the  truck 
that  delivers  our  groceries  causes  changes  in  our  nutrition”  (Clark,  1983,  p.445). 

In  1991  Kozma  published  an  article  on  Review  of  Educational  Research  - “Learning  with  media,”  that 
brings  more  fire  into  the  debate.  The  article  discusses  learning  and  media  as  complementary  processes  “within 
which  representations  are  constructed  and  procedures  performed,  sometimes  by  the  learner  and  sometimes  by  the 
medium”  (Kozma,  1991,  p.l79).  This  is  one  of  the  most  influential  critiques  of  Clark's  arguments. 

Although  the  Clark  and  Kozma  debate  really  did  begin  as  a debate,  with  a very  adversarial  atmosphere  and 
defensive  positions  taken  by  both  researchers,  the  addition  of  several  other  authors  helped  to  open  the  debate  toward 
a collaborative  dialogue.  This  was  accomplished  through  questioning  of  assumptions  (bringing  them  into  the  open), 
broadening  of  perspectives  (there  are  more  than  just  two  opposite  viewpoints),  and  redefining  the  expected 
outcomes  (creating  creativity).  The  addition  of  others  who  were  willing  to  share  their  thoughts  in  an  open  and  candid 
manner  has  assisted  in  moving  the  debate  more  toward  a dialogue  (Eigure  1). 


EIGURE  1 - Why  Clark  and  Kozma? 
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Comparing  arguments 

According  to  Clark,  it  is  not  the  media  that  influences  learning  effectiveness,  but  the  instructional  method 
(media  and  method  are  two  distinct  identities).  “An  instructional  method  is  any  way  to  shape  information  that 
compensates  for  or  supplants  the  cognitive  processes  necessary  for  achievement  or  motivation”  (Clark,  1991,  p.35). 

The  media  and  the  systems  of  symbols  associated  with  them  only  offer  "operational  vehicles  for  methods 
that  reflect  the  cognitive  processes  necessary  to  perform  a given  learning  task"  (Clark,  1983,  p.454).  It  is  possible  to 
accomplish  the  same  levels  of  performance  by  utilizing  a variety  of  media  and  their  correspondent  symbolic 
elements.  He  addresses  the  media  attribute  argument  (e.g.  “zooming  in”  in  movies  and  television)  by  referring  to  the 
fact  that  an  array  of  media  attributes  can  accomplish  the  same  learning  goals.  This  is  what  empirical  evidence  seems 
to  imply.  Kozma  contra-argues  by  stating  that  Clark’s  view  of  media  as  "delivery  trucks"  generates  an  "unnecessary 
schism  between  medium  and  method"  (Kozma,  1991,  p.205).  He  proposes  an  alternative  argument  that  media  can 
influence  learning.  He  says  that,  “some  students  will  learn  a particular  task  regardless  of  the  delivery  device.  Others 
will  be  able  to  take  advantage  of  a particular  medium’s  characteristics  to  help  construct  knowledge”  (Kozma,  1991, 
p.205). 

According  to  Kozma,  media  can  have  an  important  role  in  learning.  His  main  arguments  are  that  media  and 
methods  are  interconnected  (contrary  to  Clark’s  argument)  and  that  the  media  interacts  with  learning.  Learning  is 
influenced  by  “cognitively  relevant  characteristics  of  media  - their  technologies,  symbol  systems  and  processing 
capabilities”  (Kozma,  1991,  p.205).  Media  and  instructional  method  have  a complementary  relationship  - they  are 
an  integral  part  of  the  instructional  design  process.  “Within  a particular  design,  the  medium  enables  and  constraints 
the  method:  the  method  draws  on  an  instantiates  the  capabilities  of  the  medium”  (Kozma,  1991,  p.205). 

Analyzing  the  debate 

The  analysis  of  the  debate  clearly  shows  that  both  of  the  researchers  follow  the  verification  model.  Both 
point  out  the  studies  and  empirical  data  that  confirm  their  own  assumptions.  The  overall  tone  is  verification  and 
adversarial. 

• “Clark  (1983)  contends  that  media  do  not  influence  learning  under  any  condition,  but  the  research  reviewed 
in  this  article  suggests  that  this  position  must  be  modified.”  (Kozma,  1991,  p.  205). 

• “When  a study  demonstrates  that  media  attributes  are  sufficient  to  cause  learning,  the  study  has  failed  to 
control  for  instructional  method  and  is  therefore  confounded”  (Clark,  1994,  p.  25). 

Even  though  the  entire  debate  between  Clark  and  Kozma  is  around  verification  of  their  own  theories  and 
does  not  focus  on  seeking  instances  in  which  their  theories  do  not  work,  a more  attentive  analysis,  however,  can 
expose  some  possibilities  for  falsification.  In  the  final  analysis,  this  debate  became,  in  our  opinion,  moved  toward  a 
start  of  dialogue.  Clark  and  Kozma  did  not  intend  to  falsify  their  own  theories,  but  they  put  their  theories  into  the 
public  arena  where  they  could  be  tested  by  others  as  their  perceptions  and  arguments  evolved. 

A close  examination  of  this  debate  can  reveal  some  glimpses  of  attempts  at  falsification  and  a willingness 
to  dialogue.  Following  are  some  examples  that  illustrate  those  attempts. 

• When  Clark  started  his  studies  on  media  research  in  1980,  he  had  the  expectation  that  “media  were  a 
significant  element  in  any  educational  reform  which  sought  achievement  gains”  (Clark,  1991,  p.35). 
However  when  he  started  looking  at  literature  that  addresses  this  issue  he  could  not  find  evidence  of  his 
expectations,  their  expectations  were  not  corroborated  by  what  he  found  in  the  literature. 

• A clear  acknowledgment  that  there  is  a confusion  on  terminologies  is  recognized  by  both  researchers: 

o “There  is  a long  history  of  a basic  confusion  between”  (Clark,  1994,  p.23)  instructional  design 
technologies  and  delivery  technologies 

o Kozma  makes  a distinction  between  attribute  as  a capability  of  a medium  and  the  variability  of  its 
use.  He  argues,  “in  the  development  of  theory,  Dubin  (1996)  defines  an  attribute  as  the  property  of 
a thing  distinguished  by  the  quality  of  being  present,  while  a variable  is  the  property  of  a thing  that 
may  be  present  in  degree  (p.  35).  The  attributes  of  a medium  are  its  capabilities;  the  capabilities  of 
a medium  are  always  present”  (Kozma,  1994,  p.l3). 

• The  need  to  search  for  “negative  evidence”  is  pointed  out  by  Clark.  Such  acknowledgment  makes  it 
possible  for  their  “debate”  to  grow  into  a “dialogue”.  His  argument  is  that  scholars  tend  to  start  with  a 
solution  and  then  look  for  the  problem  that  can  be  fixed  with  this  solution: 

o “Part  of  the  problem  in  my  view  is  that  we  tend  to  encourage  students  (and  faculty)  to  begin  with 
educational  and  instructional  solutions  and  search  for  problems  that  can  be  solved  by  those 
solutions.  Thus,  we  begin  with  an  enthusiasm  for  media,  or  individualized  instruction,  or  de- 
schooling - and  search  for  a visible  contejt  in  which  to  establish  evidence  for  our  solution  (...) 

We  need  a great  appreciation  for  negative  evidence  and  to  begin  with  a focus  on  problems  (...) 
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and  then  search  relevant  research  literature  for  robust,  research -based  theories  about  the  variety  of 
solutions  to  those  problems”  (Clark,  1991,  p.39). 

• A lack  of  consistency  with  empirical  data  provided  instances  of  falsification.  This  seems  to  be  the  basis  of 
the  argument  done  by  Cunningham  (1986)  about  Clark’s  lack  of  empirically -based  claims  that  instructional 
methods  are  responsible  for  achievement  gains: 

o “Of  course  the  data  from  any  particular  comparison  may  show  no  statistically  significant 
difference  between  the  methods/media,  but  the  basic  question  (...)  is  designed  to  identify 
differences.  Thus,  we  speak  of  significant  differences,  not  significant  equivalences;  after  all,  we 
can  never  prove  the  null  hypothesis”  (Cunningham,  1986,  p.4). 

• Another  instance  of  falsification  is  pointed  out  by  Cunningham  (1986),  who  referred  to  the  nature  of 
inquiry  by  stating  that  we  should: 

o “not  make  the  mistake  of  reifying  our  data,  of  assuming  that  the  data  are  simply  measures  of 

reality  that  our  theories  will  take  account  of  eventually.  We  must  make  more  modest  claims  about 
the  validity  of  our  data  and  not  be  surprised  when  what  we  once  regarded  as  truth  changes 
dramatically.  Our  interventions  must  be  contextualized  to  particular  times  and  places  and  within 
particular  theoretical  perspectives.  (...)  if  we  come  to  understand  the  constructive  nature  of  our 
knowledge,  we  can  make  moE  modest,  context  -dependent  claims  about  the  “truth  of  the  matter” 
and  be  less  embarrassed  when  we  are  wrong.  In  other  words,  there  are  no  good  guys  and  bad  guys, 
only  guys.”  (Cunningham,  1986,  p.6/7). 

The  process  of  dialogue 

So,  what  would  an  ideal  intellectual  dialogue  look  like?  The  key  concept  of  Popper’s  philosophy  of  science 
is  “objectivity,”  which  he  believes  can  help  exercise  control  against  the  prejudice  inherent  in  subjective  knowledge 
or  feeling  of  conviction  (Stokes,  1998).  In  Popper’s  view,  the  maintenance  of  objectivity  is  central  to  an  open 
society  and  the  critical  tradition  that  fosters  social  values  such  as  free  discussions,  and  practices  such  as  publications 
and  conferences  (Stokes,  1998).  In  order  to  increase  “objectivity,”  each  party  needs  to  state  their  convictions  up 
front.  Popper  argues  that  individual  researchers  should  adhere  to  the  values  of  critical  thinking  and  discipline, 
attaining  the  publicity  of  method  (Stokes,  1998). 

We  recommend  that  the  dialogue  follow  an  intentional  framework,  providing  structure  for  analysis  and 
deleting  personal  assaults  from  the  conversation.  In  this  manner,  dialogue  would  not  be  an  attack  on  the  author  of  a 
theory;  it  would  not  be  centered  on  opinions,  but  would  be  an  effective  way  to  begin  talking  about  essential 
foundation  pieces  of  the  field  and  engage  in  intellectual  exercise. 

First,  the  foundation  of  any  successful  dialogue  is  mutual  respect.  This  includes  respect  for  people  as  well 
as  their  ideas.  Secondly,  we  would  suggest  that  the  goals  for  the  dialogue  should  be  clearly  stated.  With  this  we  do 
not  mean  that  the  goals  cannot  evolve  throughout  the  process  of  dialogue.  However,  if  the  goals  cannot  be  stated 
clearly  or  are  personal  in  nature  (“I  want  to  prove  him  wrong”),  then  perhaps  the  responses  are  not  worth  the  time  or 
effort,  as  they  will  not  build  the  field.  Thirdly,  a framework  of  explicitly  stated  assumptions  and  clear  definitions 
would  be  called  for.  One  of  the  biggest  problems  in  dialogue  is  the  use  of  terms  that  are  not  clearly  understood  by 
the  audience  or  participants.  Fourth,  openness  to  criticism  and  willingness  to  “hear”  differing  perspectives  can 
bolster  the  dialogue,  making  it  more  robust.  Finally,  if  this  process  were  placed  in  a shared  space,  where  each  voice 
could  be  heard,  dialogues  would  be  made  richer  and  more  accessible  to  practitioners  as  well  as  researchers. 

Creating  an  environment  for  successful  dialogue  begins  with  parties  who  agree  on  a platform  of  mutual 
respect.  Without  a level  of  mutual  respect,  there  can  be  no  dialogue,  only  argument.  The  underlying  assumptions  in 
a debate  are  that  one  person  is  right,  one  is  wrong,  and  that  I am  right,  therefore,  you  must  be  wrong.  This  leads  to  a 
lack  of  respect  for  each  other’s  ideas  and  in  the  end,  blocks  contributions,  which  ultimately  limits  solutions  and 
ideas.  If  the  field,  as  some  have  suggested  is  in  its  infancy,  then  perhaps  acting  in  selfish,  childish  ways  can  be 
excused.  If  we  wish  to  advance  the  field  we  need  to  develop  a willingness  to  cooperate  and  collaborate.  “Keeping 
the  spirit  of  mutuality  and  respect  alive  as  we  converse  about  our  different  views,  then,  is  a way  of  allowing  all  the 
differences  - which  are  valuable  to  creating  the  best  thinking  - to  come  to  the  surface  and  be  carefully  considered” 
(Stewart,  n.d.). 

Goals  for  the  dialogue  are  also  essential,  as  a means  of  keeping  people  and  ideas  focused.  Fierke  and 
Nicholson  (2001,  p.9),  although  taking  divergently  discrete  approaches  to  games,  clearly  state  the  purpose  of  their 
dialogue,  which  is  to  “enrich  the  discussion  of  the  role  of  games  in  international  relations,  while  elucidating  and 
recording  the  differences  between  the  two  distinct  approaches.”  This  is  not  only  helpful  to  the  reader,  but  throughout 
their  dialogue,  it  is  a maintained  focus.  In  the  end,  they  find  that  their  dialogue  has  “strengthened  each  (perspective) 
by  providing  a basis  for  making  explicit  that  which  is  implicit  in  both.”  (Fierke  and  Nicholson,  2001,  p.25)  Having 
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clear  goals  for  every  dialogue  can  assist  the  participants  in  collaboratively  building  the  IT  field  by  maintaining  focus 
on  the  bigger  picture  instead  of  the  smaller,  more  personal  one. 

Contingent  to  that  is  the  notion  that  there  must  be  an  understanding  of  the  underlying  assumptions  of  a 
theory.  This  can  be  made  easier  when  the  theorist  makes  those  assumptions  (and  biases)  explicit  to  the  reader. 
However,  readers  are  frequently  left  to  deduce  assumptions  from  the  conclusions  or  arguments  in  a paper.  In  regards 
to  the  Clark  and  Kozma  debate,  Schrock  (1994,  p.49)  states,  “his  [Claik’s]  assertions  do  rely  on  definitions  and  his 
definitions  imply  assumptions  that  may  be  worth  examining  in  great  detail.”  The  main  problem  with  assumptions  is 
that  they  are  often  implicit  and  even  the  author  is  not  aware  of  them,  making  it  that  much  harder  on  the  reader  to 
understand  the  perspectives  or  conclusions.  Still,  one  of  the  values  of  a good  dialogue  is  that  it  can  assist  in  making 
the  implicit  explicit.  It  should  be  noted  that  no  one  is  making  a judgment  about  the  rightness  or  wrongness  of 
assump  tions,  merely  arguing  that  they  need  to  be  clear  to  the  “public.” 

Another  one  of  the  most  daunting  challenges  to  creating  a successful  dialogue  is  the  prolific  number  of 
definitions  utilized  in  current  research.  Many  terms  have  not  be  fully  operationalized  or  made  explicit  to  the  reader. 
Some  fuzzy  terms  are  also  used  interchangeably,  adding  to  the  confusion.  “Part  of  what  makes  the  debate  difficult  is 
that,  while  the  participants  often  use  the  same  words,  the  words  do  not  appear  to  have  identical  meanings”  (Schrock, 
1994,  p.49).  In  addition,  Schrock  (1994,  p.49)  states,  “the  debate  is  not  simple.  It  has  emotional  and  political 
elements  and  some  nasty  definitional  ambiguities.”  Researchers  should  be  encouraged  to  define  of  their  terms, 
constraints  and  the  framework  of  their  analyses  so  that  the  readers  will  be  able  to  understand  the  logic  behind  the 
interpretation.  It  must  be  noted  that  the  more  explicit  the  definitions,  the  easier  it  will  be  for  the  practitioner  to  utilize 
the  theory.  Seels  (1997)  states  that  the  use  of  definitions  could  facilitate  theory  generation  by  improving 
communication  within  and  outside  the  IT  field.  In  addition,  definitions  create  identity,  which  contributes  to  the 
maturity  of  the  field. 

Openness  to  criticism  and  a willingness  to  “hear”  all  of  the  sides  go  hand  in  hand.  In  the  Clark  and  Kozma 
debate,  it  appears  that  the  reader  is  left  to  decide  between  one  or  the  other,  when  both  stances  have  merit  and 
problems.  Each  individual  author  “feels  that  his  position  is  in  the  interest  of  prescriptive  science”  (Schrock,  1994, 
p.5 1),  when  in  reality,  they  both  share  many  of  the  same  concerns.  In  examining  the  debate,  it  appears  that  “Clark 
could  define  the  debate  in  a way  that  would  always  allow  him  to  win”  (Schrock,  1994,  p.49),  but  then,  so  could 
Kozma.  In  a fruitful  dialogue,  participants  would  listen  carefully  and  acknowledge  the  strengths  and  weaknesses  in 
order  to  move  jointly  to  a new,  stronger,  shared  perspective. 

The  notion  of  “shared  space”  in  a dialogue  (Boone,  2001)  does  not  mean  that  those  involved  in  the  dialogue 
have  to  be  sitting  in  the  same  room.  They  could  be  in  an  electronically  wired  space,  or  even  in  a shared  print  space. 
Or  at  a conference,  talking  together  face-to-face.  The  important  point  is  that  everyone  is  together.  From  this 
perspective,  print  media  is  one  of  the  most  available  methods  of  disseminating  and  bringing  information  to  the 
public  in  a shared  space.  Journals  should  publish  entire  dialogues,  not  merely  one  side  or  unrelated  articles. 
Currently,  most  dialogues  must  be  carefully  researched  in  order  for  a reader  to  follow  the  discourse.  Just  for  this 
team  of  researchers  to  gather  the  articles  necessary  for  this  paper  took  hours  and  hours.  Most  people  do  not  have  or 
will  not  spend  the  time  necessary  to  get  a complete  picture  of  the  dialogue.  Journals  should  consider  different 
layouts,  such  as  publishing  dialogues  side-by-side,  which  would  make  the  conversation  both  more  accessible  to 
readers  and  simpler  to  follow.  The  publication  of  debates  or  dialogues  in  the  field  should  not  be  exceptional,  but  part 
of  a discipline  featured  in  a “shared  space”. 

It  is  also  vital  that  language  used  by  theorists,  including  explicit  examples,  be  accessible  to  readers, 
allowing  them  to  freely  join  the  conversation  and  assist  in  creating  a feedback  loop  between  theorists  and 
practitioners.  Convention  planners  should  also  consider  designing  sessions  that  invite  this  kind  of  dialogue  and 
welcome  both  theorist  and  practitioners  alike.  The  more  these  groups  can  be  together,  discussing  the  theories  and 
best  practices  of  the  field,  the  richer  the  IT  field  will  become,  to  everyone’s  mutual  benefit. 

Conclusions 

We  admit  this  to  be  a work  in  progress,  but  feel  that  our  ideas  and  research  have  been  in-depth  and 
thorough.  By  focusing  on  a single  example  of  a debate  and  dialogue  we  were  able  to  examine  it  in  depth  and 
comprehensively.  The  Clark  and  Kozma  debate  has  provided  a gold  mine  of  instances  of  verification  and/or 
falsification  and  we  will  continue  to  look  at  different  aspects  of  that  debate  as  we  move  forward  with  our  project. 

In  addition,  the  Popperian  lens  of  falsification  along  with  the  Lakatos’  perspective  provided  a useful 
background  for  our  analysis  of  the  debate  and  supported  the  foundational  ideas  of  dialogue  as  well,  thus  providing  a 
consistent  method  for  viewing  the  construction  of  our  guidelines.  Although  this  is  an  initial  paper  we  hope  that  it 
will  foster  an  awareness  of  the  need  for  dialogue.  Future  developments  can  continue  to  refine  the  process  of  dialogue 
to  provide  a mean  for  improving  the  discipline  of  the  dialogue  for  the  purpose  of  informing  the  IT  field. 
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Abstract 

Our  physical  teaching  world,  once  filled  with  desks  and  blackboards,  has  become  a virtual  teaching  world 
filled  with  computers  and  software.  Walt  Disney  Pictures  entertained  everyone  with  "The  Emperor's  New  Groove" 
but  as  students  and  instructors  get  into  Groove  software,  entertainment  is  replaced  by  diligent,  intense  work  as  they 
learn  processes  to  create  a new  environment.  Is  there  a "happily -ever  after”  with  this  learning  experience?” 

Introduction 

One  of  the  goals  of  the  Purdue  Program  to  Prepare  Tomorrow’s  Teachers  to  Use  Technology  (P3T3)  is  to 
provide  technology -rich  environments  for  faculty  that  will  provide  a model  for  future  teachers.  As  participants, 
faculty  members  attend  a "start-up"  workshop  to  introduce  them  to  the  project,  and  to  technology  in  the  classroom. 
At  the  conclusion  of  the  "start-up"  sessions,  faculty  members  develop  and  share  personal  technology  integration 
plans.  One  of  the  plans  conceived  by  a faculty  member  in  Mathematics  education  was  to  incorporate  a tool  in  the 
classroom  that  would  enable  students  from  the  local  Purdue  site  to  collaborate  with 

? students  in  classrooms  in  other  universities  (nationally  and  worldwide), 

? other  faculty  members,  and 
? consultants  in  the  field. 

Pervious  attempts  to  use  traditional  distance  education  media  such  as  videoconferencing  using  CU-SeeMe, 
Polycom,  and  WebCT  provided  limited  capability  to  meet  the  goals  of  the  course.  As  with  any  desire  to  change  the 
design  of  a course,  there  are  successes,  failures,  and  challenges  to  meet  the  needs  of  the  students  - especially  when 
technology  is  included  in  the  changes  that  are  made.  Throughout  this  paper  we  will  compare  and  contrast  Groove 
with  other  technologies,  document  skills  and  attitudes  needed  to  attempt  distance  education  using  technology  tools, 
and  provide  information  about  other  lessons  we  learned  along  the  way.  It  is  our  hope  that  through  this  paper  we 
might  provide  you  with  insights  if  you  consider  using  a collaborative  tool  to  expand  your  realm  of  educational 
possibilities  so  you  can  experience  a “happily -ever  after”  with  your  students. 


Theoretical  Framework 

Groove  software  - a Peer-to-peer  networking  system 

Peer-to-peer  (P2P)  computing  is  “currently  touted  as  the  next  ‘killer  application’  for  the  Internet”  (Gartner 
Consulting,  2001).  Peer-to-peer  means  that  the  PC  and  the  server  are  one  and  the  same  thing.  Clay  Shirky,  a leading 
figure  in  the  P2P  community,  stated  “peer-to-peer  is  a swathe  of  applications  that  harness  resources  at  the  far  edges 
of  the  Internet,  where  the  machines  have  complete  freedom  (or,  at  least,  significant  autonomy)  from  the  central 
servers”  (Economist,  2001,  p.  21).  P2P  technology  is  designed  to  enable  shared  resources.  It  provides  a way  for 
users  to  link  together  without  the  use  of  a server  and  facilitates  data  sharing,  resource  sharing,  and  workgroup 
collaboration.  P2P  can  be  compared  with  the  concept  of  the  World  Wide  Web  - a system  of  hardware  and  software 
that  has  revolutionized  the  ability  to  find,  store,  and  share  information.  At  the  same  time,  the  Web  can  be  compared 
with  the  use  of  a dictionary  or  an  encyclopedia.  It  is  a place  where  information  and  definitions  can  be  found,  but  in 
order  to  find  the  information,  you  must  have  access  to  the  resource  itself  (i.e.,  the  book).  With  P2P  networking,  each 
user’s  computer  has  a full,  personal  copy  of  the  resource  media.  But  when  used  in  the  classroom,  the  instructor  must 
apply  disciplinary  measures  to  the  use  of  the  media;  it  cannot  simply  be  a place  of  storage.  “To  create  a working 
knowledge  community,  users  must  make  content  available  and  functional,  and  community  members  must  take  the 
time  to  search  for  content,  clarify  information,  and  provide  feedback  as  to  its  usefulness  (Hofmann,  2002).” 

Groove  software  is  one  of  the  many  P2P  software  programs  designed  to  provide  solutions  to  today’ s needs 
to  collaborate  and  integrate  partners  in  the  process  of  education.  In  an  article  in  Business  Week  On-line  Ray  Ozzie 
states,  “A  recent  Media  Metrix  report  indicates  the  number  of  at- work  users  of  Internet  Messenger  has  increased  by 
22%  in  the  past  six  months . . . [this  will]  drive  adoption  of  peer-to-peer  collaboration  technologies  such  as  Groove” 
(Kharif  and  Salkever,  2001).  Groove  consists  of  a comprehensive  collaboration  platform.  The  client,  called  the 
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transceiver,  includes  a large  variety  of  standard  modules  for  discussion  groups,  P2P  and  server-based  shared  files,  a 
sketchpad,  voice  and  text  chat,  and  scheduling  (Yager,  2001).  Groove  tools  include  real-time  communication  using 
chats,  file  sharing,  browsing  together  on  the  Internet,  and  instant  messaging.  These  tools  can  be  used  is  such  a way 
that  information  can  be  archived  in  one  shared  space,  improving  file  management  and  feedback  capabilities. 


WebCT 

Olsen  (2001)  indicated  WebCT  is  much  more  widely  used  in  educational  settings  than  Groove.  It  consists 
of  four  major  functional  areas:  course  content,  course  management,  student  tools  and  communication  tools.  It  is  a 
productive  resource  as  long  as  you  understand  the  creator’s  philosophy  of  an  instructor’s  needs  to  teach  a course. 

The  instructor  places  their  own  course  data  into  WebCT’ s interface  and  for  the  most  part,  it  is  ready  to  share  with 
students.  It  is  not  easy  to  set  up  the  software  to  allow  outside  instructors  of  professionals  to  be  a visitor/participant 
for  one  or  two  sessions  only.  The  key  issues  for  focus  in  the  WebCT  environment  are  that  information  originates 
from  one  source  (the  instructor)  and  it  should  be  monitored  (time  on-line,  quizzes,  grades),  as  in  the  face-to-face 
classroom  setting.  It  is  assumed  that  the  syllabus  doesn’t  change,  that  the  instructor  wants  to  hold  on-line  quizzes, 
that  there  is  one  teacher  and  many  students.  It  is  also  fairly  static  - once  information  is  added  by  the  instructor  it  is 
assumed  that  it  will  not  change  since  it  requires  a new  upload  of  files  each  time  there  is  a change.  For  instructors 
who  use  this  philosophy,  WebCT  will  provide  an  adequate  mold  for  traditional  distance  education  even  though  there 
are  some  problems  associated  with  the  software.  For  example,  the  interface  design  of  WebCT  is  not  intuitive  (S.  S., 
personal  communication.  May  7,  2002).  Considering  the  tracking  student  function,  Olsen  (2001)  indicated  that  the 
data  gathered  by  the  tracking  system  lacks  sophistication,  and  users  largely  ignore  the  data.  And  the  threaded 
discussion  in  the  discussion  board,  LaMaster  & Morley  (1999)  found  that  the  way  the  postings  were  listed 
chronologically  did  not  cause  students  to  think,  and  students  felt  that  interactions  on  the  bulletin  board  were  “like  a 
chore.” 

Whereas  WebCT  uses  a teaching  philosophy  for  its  design.  Groove  uses  a business  collaboration 
philosophy.  It  is  assumed  that  nothing  is  static  - there  is  need  for  constant  change.  Groove  has  many  of  the  same 
features  as  WebCT:  discussion  boards,  calendar,  chat  room,  instant  messages,  and  file  sharing.  It  does  not  have  on- 
line quiz  capabilities  or  grade  posting  (although  it  certainly  could  be  programmed  to  provide  these  things);  but  it 
does  have  other  tools  that  are  important  to  the  project.  It  has  the  means  to  harness  time  outside  of  the  classroom  by 
facilitating  collaboration  on  projects  and  participants  can  be  added  at  any  time  so  that  multi-site  project-based 
learning  and  coordinated  workshops  are  possible. 


Methodology 

Who  were  we? 

Researchers  consisted  of  two  P3T3  graduate  assistants.  One  was  assigned  to  the  faculty  member  as 
technical  support,  the  other  as  a research  assistant.  The  researchers  attended  class  each  week  and  kept  intricate 
records  of  student  comments  and  interactions  with  each  other  and  the  professor.  In  addition,  surveys  and  interviews 
were  used  to  explore  student  attitudes  and  comments  for  Groove-mediated  learning  environment. 


What  did  we  Use? 


Hardware 

In  the  classroom  a wireless  Internet  system  and  Laptop  computers.  Outside  of  the 
classroom  the  researcher’s  computer  served  as  a primary  resource  for  file  sharing  and 
shared  spaces,  with  the  professor’s  computer  as  a backup.  Students  also  used  a personal 
computer  in  their  homes. 

Software 

Groove  Preview  Edition  version  1.2,  free  from  http://Groove.net. 

Support 

The  researcher  served  as  primary  contact  for  hardware/software  support.  The  IT  staff  in 
the  School  of  Education,  and  Groove  technical  support  and  forums  were  also  used. 
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How  Was  Information  Distributed  and  Collected? 

Students  accessed  the  syllabus  via  a webpage  linked  in  one  of  the  shared  spaces.  Resource  materials  (such 
as  the  book  chapters)  were  provided  in  an  asynchronous  file-sharing  tool  or  via  an  instant  message.  Students 
collaborated  synchronously  and  asynchronously  on  projects  using  the  discussion  board  and  posted  their  projects 
there  as  well.  Synchronous  chatting  on-line  and  Via  Video  (a  video  and  audio  transmission  tool)  were  used  for 
students  who  were  attending  the  synchronous  learning  sessions  from  a distance.  The  researchers  attended  each  class 
during  the  semester  and  analyzed  student  verbal  and  non-verbal  reactions  to  the  use  of  P2P  networking. 

Qualitative  data  were  gathered  from  five  sources: 

1)  the  first  online  survey  conducted  during  the  week  (collecting  students’  perceptions  of  the 
advantages  and  disadvantages  of  Groove), 

2)  the  second  online  survey  conducted  in  the  last  class  of  the  semester  (examined  students’ 
perceptions  about  the  specific  functions  of  Groove), 

3)  follow-up  interviews  conducted  with  four  participants  and  the  graduate  assistant  of  the  class 
(focused  on  participants’  perceptions  of  differences  between  Groove  and  WebCT), 

4)  the  researchers’  discussions  with  student  participants,  and 

5)  observations  of  students’  responses  during  the  semester  as  noted  and  archived  by  the  researchers. 

Groove  Integration 

Successes 

Class  sessions  were  held  once  a week  in  the  Twenty-first  Century  Conceptual  Tools  (TCCT)  classroom 
located  on  campus.  The  photo  on  the  right  shows  how  the  classroom  is  designed  to  enhance  student  and  instructor 
use  of  computers  in  the  classroom  through  the  use  of  an  open  architecture.  During  the  class  sessions,  the  instructor 
chose  a seat  that  allowed  the  best  interaction  with  students.  At  times  this  was  near  the  whiteboards  to  use  for 
sketching,  while  at  other  times  it  was  at  the  rear  of  the  classroom  to  allow  guests  to  see  him  during  his  lecture  as 
they  “attended”  class  using  the  ViaVideo  teleconferencing  system  located  at  the  television.  Typically  the  professor 
opened  the  class  session  to  questions  about  the  assigned  readings.  After  questions  were  answered,  or  if  there  were  no 
questions,  he  would  expand  on  the  content  in  the  readings  or  discuss  application  of  content.  Lectures  were  kept  at  a 
minimum  - a forum  of  discussion  was  preferred.  If  conversation  was  slow,  or  when  many  ideas  were  discussed  and 
the  class  was  ready  to  move  on  to  new  concepts  the  professor  would  ask  students  to  use  laptop  computers  (provided 
for  each  student)  and  encourage  students  to  share  ideas  in  a discussion  board  and  comment  on  postings  made  by 
others.  This  allowed  students  to  convey  their  own  thoughts,  communicate  them,  and  gather  feedback  from  other 
students  about  their  ideas.  At  specific  times  student  also  posted  their  Model  Eliciting  Activity  (MEA)  projects  in 
Groove  and  solicited  feedback  from  others  about  their  ideas  and  the  processes  they  used  for  instruction.  This  was 
also  done  during  class  time.  After  class  sessions  were  complete,  discussions  continued  inside  of  the  network  and 
outside  of  class  in  asynchronous  style.  The  professor  often  would  pose  a question  taken  from  a student  response  (or 
question)  that  was  found  in  the  previous  class  on-line  discussion. 

Throughout  the  semester,  the  researchers  also  used  Groove  to  determine  if  a student  was  online.  Groove 
provides  a visual  icon  to  indicate  if  a user  is  active  on  their  computer.  Since  part  of  the  research  was  designed  to 
provide  a preliminary  look  at  the  P2P  tools  rather  than  as  a formative  evaluation,  the  researchers  generally  did  not 
use  a set  time  or  target  a specific  student.  Instead  they  attempted  to  gather  a sample  of  information  from  a variety  of 
students  at  various  times.  Here  is  a sample  of  the  kind  of  conversation  that  occurred  during  the  semester: 

Researcher:  1/30/02  4:52  PM 

Let's  try  an  experiment  if  you  have  the  time. ..Yes? 

Student:  1/30/02  4:55  PM 

Well,  thanks  for  being  so  patient  with  me  (with  all  of  us)  as  we  adjust  to  this  groove  thing.  Yes  I can 

try  your  experiment 

Researcher:  1/30/02  4:54  PM 

I'm  going  to  send  you  a new  invitation.  Give  me  a minute.  Notice  on  the  left  column  that  Tm  going  to 

leave  the  "Active"  space  (just  a tip  for  future  use)... 

As  noted  by  the  student,  there  was  some  difficulty  transferring  class  work  to  the  Groove  setting,  but  the 
instructor  provided  a great  deal  of  assistance,  technical  and  time,  for  them  to  adjust  to  the  new  setting.  Both 
researchers  believe  this  is  a key  component  of  designing  a course  when  new  technology  is  a principle  factor. 

Ultimately,  the  researcher  made  the  following  conclusions: 
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• The  instructor  can  have  more  ownership  of  the  class  with  Groove  when  compared  with  Web  CT 
because  Groove  does  not  require  a central  server. 

• Learning  extends  beyond  the  classroom  activities.  Using  real-time  communication,  Groove  provides 
students  have  an  opportunity  to  learn  from  each  other.  This  can  happen  inside  or  outside  of  the 
classroom.  Two  of  the  students  indicated  they  preferred  to  share  their  thoughts  with  classmates  in  real- 
time communication  as  compared  with  discussion  boards. 

• Management  of  information  is  improved.  In  Groove  environment,  students  are  able  to  create  their  own 
shared  space  without  the  supervision  of  the  instructor.  One  student  stated,  “Shared  space  is  helpful  to 
keep  your  tasks  separated.’’ 

• Interface  design  is  user-friendly.  During  an  interview,  one  student  said,  “In  Groove’s  learning 
environment,  it’s  much  easier  to  navigate  through  the  windows,  you  have  the  various  tabs  all  within 
one  screen.’’ 

• Multitasking  opens  new  areas  for  research  and  functionality.  During  the  interview,  one  student  said  she 
could  do  multi-tasking  in  Groove.  For  example,  she  could  do  a chat  with  one  student  and  at  the  same 
time  send  information  to  a student  at  home.  This  was  perhaps  the  greatest  discovery  for  the 
researchers.  Students  are  familiar  with  multi-tasking.  They  talk  on  the  telephone,  watch  television,  and 
type  homework  at  the  same  time.  Using  chat  rooms  on-line,  they  control  more  than  one  conversation  at 
a time.  Yet  the  traditional  classroom  remains  focused  on  full  attention  to  only  one  form  of 
communication  at  a time,  whether  it  is  a lecture,  a video,  or  a reading  assignment.  The  truth  is,  in  the 
traditional  classroom  students  may  take  notes  and  appear  to  transcribe  the  lecture  - but  they  are,  at  best, 
writing  notes  that  incorporate  their  own  thought  process  and  relate  the  information  to  their  own 
systems  of  knowledge. 

In  the  classroom  during  synchronous  learning  sessions,  students  may  have  technology-mediated 
communication  with  multiple  people  while  remaining  task-focused.  The  simplest  example  of  this  was 
experienced  at  the  beginning  of  the  semester  when  students  used  laptops  in  the  classroom.  The 
technology  assistant  was  available  in  the  classroom  during  the  discussion.  Without  disturbing  the  class, 
students  would  use  Groove  instant  messages  to  inform  the  assistant  of  problems  (such  as  low  batteries 
or  software  glitches).  The  assistant  responded  to  the  problems  on  an  individual  basis  with  minimal 
interruption  to  others.  Students  became  adept  at  doing  the  same  kinds  of  things.  One  student  shared, 
“Being  able  to  use  Groove  independently  fromthe  Professor  in  class  is  a real  opportunity  to  learn  from 
others  and  feel  more  comfortable  about  sharing  thoughts  with  classmates  of  the  same  level  of 
understanding.” 

Due  to  unforeseen  circumstances,  two  students  entered  one  class  session  remotely.  Both  attended  class 
using  Groove,  although  one  also  linked  using  Polycom.  Audio  from  the  local  site  was  available  with 
the  Polycom  unit,  but  remote  audio  did  not  transmit.  The  student  immediately  used  a chat  function  in 
Groove  asking  for  support  and  determined  the  problem  could  not  be  fixed  before  the  start  of  the  class. 
Groove  chatting  was  used  to  communicate  the  information  instead.  Both  remote  students  entered  the 
chat  area  with  the  technical  assistant  who  transcribed  the  lecture  for  them.  At  the  same  time,  the 
support  assistant  had  to  manage  problems  in  the  classroom.  There  were  times  that  instant  messages  or 
chats  from  other  students  were  opened  and  responded  to  without  the  remote  students’  being  aware  of 
the  interruption.  At  one  point,  the  technical  assistant  was  managing  3 conversations  at  the  same  time 
while  still  transcribing  lecture  notes.  This  might  be  considered  a remote  instance  of  multi-tasking;  but 
both  remote  students  reported  that  they  were  also  joining  in  smaller  chats  with  students  at  the  local  site 
as  well.  One  remote  student  was  very  excited  about  this  capability,  as  it  challenged  her  to  remain 
involved  in  the  discussion  and  projects  that  were  developing  at  the  local  site  rather  than  feeling  isolated 
and  uninvolved.  It  was  natural  for  her  to  manage  multiple  tasks  at  the  same  time  and  Groove  facilitates 
this  very  well. 

Challenges 

Groove  is  computer  dependent.  If  a student  uses  more  than  one  computer  (and  most  do),  the  software  must 
be  loaded  on  each  one.  The  student  must  also  use  identities  unique  to  each  computer  so  that  others  can  determine 
which  location  they  want  a message  to  send  a message.  For  example,  one  of  the  researchers  has  4 accounts  which 
are  differentiated  as  follows: 

? her  name_office, 

? her  name_class 
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? her  name_laptop 

? her  name_home 

If  a student  wanted  to  contact  the  researcher,  he/she  would  look  at  the  list  of  users  on-line  first  to  see  if  the 
researcher  was  active.  If  not,  timing  would  dictate  where  the  message  would  be  delivered,  e.g.  after  working  hours, 
messages  were  sent  to  the  home  account.  Switching  machines  is  a real  inconvenience.  Shared  spaces  cannot  be 
accessed  without  an  invitation  from  an  active  participant,  and  computers  depend  upon  one  another,  so  if  a person  is 
off-line,  data  may  not  be  transferred  when  it  is  needed  by  the  recipient.  One  student  complained,  “There  were  times 
this  year  when  I was  traveling  and  missed  a lot  of  stuff  in  Groove  simply  because  I didn’t  have  access  to  it.  It  also 
took  up  a lot  of  space  on  my  computer.” 

To  use  Groove  efficiently  and  effectively,  RAM  is  an  issue  on  older  computers  that  are  not  upgraded.  It 
also  requires  a fast  Internet  connection  or  file  downloads  can  keep  the  user  on-line  for  an  hour  or  more  simply 
transferring  files.  During  chats,  the  faster  computers  will  “log”  messages  more  quickly  and  keep  the  user  in  the 
conversation  with  a real-time  presence.  Slower  computers  cause  a delay  in  posting  messages  in  the  chat  and  could 
cause  the  message  to  appear  in  the  wrong  place  in  a conversation. 

Even  though  multi-tasking  comes  natural  to  us,  one  student  felt  use  of  the  computer  to  “pass  notes”  without 
stopping  the  class,  was  still  an  interruption.  Students  are  aware  of  computer  keyboards  being  used,  and  eyes  that 
aren’t  on  the  speaker  during  class  discussions  or  lectures.  For  this  student  it  was  distracting  and  irritating  - they 
perceived  the  others  who  used  the  keyboard  were  not  paying  attention  or  “telling  secrets.”.  One  student  who  enjoyed 
multitasking  even  offered  their  own  problems  using  the  “quiet  conversations”  in  the  background  saying,  “...it  also 
allows  for  some  distractions,  especially  when  those  chats  turn  to  other,  unrelated  topic.” 

Comparison  with  WebCT 

Groove  and  WebCT  are  built  on  different  philosophies.  The  professor  said,  “Groove  was  designed  for 
business  settings.  Groove  encourages  informal,  unmonitored,  off-line  communication  and  provides  more  open 
environment.  However,  WebCT  was  designed  for  instructional  settings.” 

There  is  a difference  in  course  design  and  management.  One  researcher’s  observations  and  discussions  with 
the  graduate  technical  assistant  who  designed  the  course  indicated  that  it  takes  minimal  technical  requirements  to 
prepare  a course  for  Groove.  At  the  same  time,  more  effort  must  be  made  to  transfer  knowledge  of  the  way  Groove 
works  on  multiple  computers  and  the  understanding  of  the  hardware  requirements  to  make  the  tool  work  for  the 
course.  The  assistant  also  indicated  that  her  experience  with  WebCT  requires  uploading  course  materials  into  a 
server  File  Manager.  Simple  additions  and  changes  require  a significant  number  of  mouse  movements  and  clicks  to 
upload  the  data. 

Technical  support  in  the  classroom  is  needed  much  more  when  using  Groove  and  videoconferencing. 
Because,  for  the  most  part,  WebCT  is  asynchronous,  and  Groove  partnered  with  videoconferencing  is  intended  to  be 
synchronous,  it  is  imperative  that  technical  assistance  is  available  for  the  occasional  glitches  and  failures.  Loss  of 
transmission  could  mean  the  delay  of  a class  with  Groove,  but  with  WebCT  students  simply  log  in  at  a later  time  to 
retrieve  information  they  need  or  read  messages  they’ve  missed  in  the  on-line  discussions. 

Students  find  the  word  processing  capabilities  in  Groove  much  simpler.  The  discussion  area  allows  bold, 
italics,  color  changes  in  text  in  a fashion  similar  to  a word  processor.  WebCT  requires  the  use  of  HTML 
programming  tags  to  change  the  font/size  characteristics. 

Instructors  find  key  components  such  as  on-line  quizzes  and  grade  books  available  in  WebCT,  but  missing 
in  Groove.  For  instructors  who  find  these  tools  useful.  Groove’s  testing/grading  process  is  much  more  difficult  and 
requires  additional  resources  to  program  the  software  to  meet  the  instructor’s  needs. 


Outcomes 

When  students  are  faced  with  opportunities  to  use  multiple  resources  to  gain  knowledge  (all  available  at 
their  desktop),  instructors  must  be  prepared  to  change  their  own  ways  of  thinking  based  upon  results  apparent  from 
the  instruction.  No  longer  does  the  instructor  need  to  be  the  major  source  of  information.  Instead  they  must  facilitate 
and  mentor  students  to  become  problem-solvers  (as  indicated  by  the  student  who  contacted  others  for  help  when  the 
researcher  was  not  available)  and  help  students  develop  higher-level  thinking  as  shown  by  the  deductions  made  by 
students  who  used  Groove  for  multkasking  and  recognized  their  own  abuse  of  the  system.  In  this  way  students  will 
leave  the  “classroom”  with  multiple  possibilities  for  learning.  Students  may  actually  develop  skills  rather  than  gain 
specific  information  - and  this  is  probably  the  best  result. 

Additionally,  students  were  asked  to  provide  feedback  when  the  research  and  projects  were  completed  via 
the  collaborative  tools  in  Groove.  Feedback  from  multiple  students  gave  a much  broader  sample  of  potential 
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problems  with  the  student’s  design  or  instruction  within  the  project.  The  diverse  responses  students  provided  to  each 
other  proved  that  this  system  is  effective  and  perhaps  even  necessary  for  coursework.  It  is  the  agreed  opinion  of  the 
researchers  that  students  find  the  collaborative  tools  a positive  means  to  share  ideas  and  generate  synergy  as  they 
create  their  course  assignments. 

Old  ways  of  explaining  concepts  via  lecture  and  text  must  be  replaced  with  new  ways  of  communicating. 
Considering  all  of  the  tools  available  in  P2P  software,  it  is  restricting  to  limit  a student’s  resources  to  text,  lecture 
and  discussion  simply  sent  via  distance  education  tools.  Even  though  these  text,  lecture  and  discussion  tools  are 
restrictive,  they  should  not  be  eliminated.  Instead  instructors  should  push  to  become  multi-tiered  designers  and 
communicators.  And  because  technology  changes  daily,  the  instructor  will  almost  certainly  be  in  a position  that  the 
course  will  always  be  in  a state  of  evolution.  It  will  probably  never  be  designed  in  such  a way  that  changes  become 
minimal  - as  they  are  when  textbooks  and  paper  handouts  accomplish  course  objectives.  But  the  success  of  this 
course  and  potential  for  future  collaboration  transform  obstacles  into  challenges. 


Recommendations 

Instructors  who  prepare  for  many  changes,  challenges  and  opportunities  when  using  technology -mediated 
communication  are  rewarded  beyond  the  success  of  their  students;  they  are  also  provided  with  insights  for  future 
research  and  exp  loration  into  a new  form  of  “distance  education.’’  Many  smaller  obstacles  may  still  remain  such  as 
time  zone  problems,  sometimes  overwhelming  weekly  discussions,  and  balancing  students  who  have  diverse  skills 
with  technology,  but  the  four  main  themes  targeted  in  this  paper  confirm  the  need  to  gather  statistical  data  and 
continue  the  research  to  determine  if  this  evolving  form  of  distance  education  can  be  successful  for  both  students 
and  instructors  and  create  a platform  for  multi-tasking,  multi-functional  systems  for  technology,  students,  and 
instructors.  Instructors  who  consider  using  this  tool  must  first  consider  their  needs  and  the  abilities  of  the  students. 
Without  proper  control  of  any  technological  tool,  students  will  be  lost  in  the  technology  and  gain  no  content  during 
the  instruction. 

Here  are  some  questions  an  instructor  should  ask  before  considering  this  software: 

? Should  I use  outside  participants  for  instructional  modules  to  enrich  my  course? 

? Do  my  students  have  the  ability  to  access  the  software  outside  of  the  classroom?  (They  cannot  depend  upon 
PUCC  labs  for  this  software  or  for  support) 

? Will  students  be  technology  proficient  enough  to  follow  on-line  help  or  instruction  (as  indicated  in  this 
research)  without  face-to-face  instruction? 

? Will  students  be  savvy  enough  with  technology  to  take  the  tools  beyond  a beginner’s  level  and  push 
themselves  to  utilize  the  strengths  of  this  system  (i.e.  collaboration,  harnessing  work  outside  of  the 
classroom,  multi-tasking,  etc.)? 

? Am  I willing  to  devote  my  time  for  supporting  the  technology  or  enlist  a support  team  for  my  students? 

If  the  answer  to  most  of  these  questions  had  been  “no,”  distance  education  tools  like  WebCT  may  be  be  the 
best  alternative  since  they  are  designed  for  instruction.  But  since  the  answer  was  “yes,”  then  P2P  networking 
provided  a source  for  further  research  and  trial.  It  has  the  capability  to  change  not  only  what  is  “done,”  but  to  also 
change  “ideas  about  learning”  as  students  challenge  instructors  with  their  abilities  to  go  beyond  course  goals  and 
objectives  into  a real  world  of  student-centered  distance  education.  And  that,  we  believe,  would  be  a new  Groove 
every  emperor  (or  instructor)  would  live  with  “happily  every  after.” 
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The  Role  of  Relevance  in  Motivation  and  Learning 

Keller  maintains  that  motivation  “is  the  neglected  heart  of  our  understanding  of  how  to  design  instruction’’ 
(1983,  p.  390),  and  in  response,  developed  the  ARCS  Model  of  Motivation  design  (1984,  1987a).  This  model  views 
motivation  as  the  combined  effects  of  attention,  relevance,  confidence  and  satisfaction.  Keller  has  supported  these 
elements  from  a rich  psychological  research  base.  The  ARCS  model  has  been  widely  embraced  by  designers  as  an 
intuitive  and  simple  tool  to  facilitate  learner  motivation.  However,  the  attention,  confidence  and  satisfaction 
components  seem  to  have  a more  robust  body  of  empirical  support  (e.g.  Bandura,  1977;  Dweck,  1986;  Fleming, 

1987;  Kanfer  & Ackerman;  1989;  Schunk,  1985),  than  does  the  relevance  component.  More  specifically,  attention, 
confidence  and  satisfaction  have  strong  links  to  learning  theory,  both  behavioral  and  cognitive.  This  paper  presents 
that  while  relevance  has  lagged  behind  the  other  ARCS  components  in  research,  its  impact  on  learning  may  be 
directly  tied  to  recently  developed  cognitive  processes. 

The  Concept  of  Relevance 

Relevance  is  a term  laden  with  many  meanings.  Most  commonly  it  implies  perceptions  of  interest,  value 
and  utility,  and  it  has  been  studied  from  each  of  these  orientations,  although  these  orientations  are  often  overlapping. 
For  example,  Herndon  (1987),  Deci,  (1992),  Schiefele,  (1991),  Small,  Dodge  & Jiang,  (1996),  and  Dabbagh,  (1996) 
all  examine  the  role  of  interest  in  their  work  on  relevance.  They  all  suggest  that  if  something  is  interesting  to  the 
learner  it  is  relevant.  For  example,  Dabbagh  (1996)  found  that  by  increasing  personal  relevance  (the  degree  to  which 
the  content  is  tied  to  previous  experiences  and  interests)  reports  of  motivation  increased,  as  did  the  time  spent 
engaged  in  the  CBI  program. 

It  is  often  difficult  to  distinguish  between  relevance  viewed  from  the  interest-orientation,  and  attention. 
Mahoney  & Lyday  (1984)  provide  a clear  example  of  relevance;  it  is  important  to  note  that  they  talk  about  this 
factor  in  terms  of  learner  interests.  This  obscure  distinction  between  "relevance”  and  "interests”  makes  it  difficult  to 
discuss  the  role  relevance  plays  in  motivation  conclusively,  for  much  of  the  content  that  interests  learners,  interests 
them  specifically  because  the  content  has  some  value  to  their  life.  It  is  often  difficult  to  determine  which  of  these 
variables  is  more  influential. 

Small,  Dodge  and  Jiang  (1996)  attempted  to  clarify  the  overlap  that  exists  among  many  of  the  ARCS 
variables.  For  example,  does  the  interest  variable  support  perceived  relevance  or  sustained  attention?  To  do  so  they 
asked  students  a series  of  question  about  what  contributed  to  their  interests  and  boredom.  Their  analysis  of  the 
responses  showed  that  attention  and  relevance  (or  a lack  of)  accounted  for  the  vast  majority  of  boring  and  interesting 
situations.  Furthermore,  students  reported  that  the  value  of  the  learning  task  greatly  influenced  motivation.  In 
addition,  when  asked  about  boring  situations  and  relevance,  students  gave  a high  rating  to  instmctional  materials. 
Indicating  that  if  the  instructional  materials  did  not  have  a high  degree  of  relevance,  they  were  boring.  . 

Value  is  also  central  to  a discussion  of  relevance.  Value  is  best  described  as  utility.  One  is  likely  to 
attribute  value  to  something  and  deem  it  relevant  if  it  is  useful.  Utility  is  a critical  aspect  of  relevance.  Studies  have 
shown  that  if  learners  perceive  the  value  of  instruction  their  motivation  will  increase.  For  example,  Bainbridge- 
Frymer  and  Shulman  (1995)  examined  relevance  and  state  motivation  (a  student’s  motivation  for  a specific  task, 
subject  at  a specific  time).  Their  examination  of  470  undergraduate  students  found  that  relevance,  as  reported  on  a 
Likert  scale  measuring  elements  of  interest  and  value,  accounted  for  a significant  amount  of  the  variance  in  state 
motivation.  In  their  study,  subjects  reported  a benefit  from  the  learning.  Indeed  they  believed  something  was  “in  it 
for  them”  and  that  there  would  be  a reward  for  their  efforts.  Indeed  rewards  are  another  way  to  assign  value. 

Value  can  also  be  discussed  within  the  framework  of  goal  orientation  (Dweck,  1986;  Dweck  & Leggett, 
1988).  Instruction  has  value  if  it  is  useful  in  aiding  one  to  reach  one’s  goal.  Indeed,  Hudson  (1988)  found  this 
indirectly  in  her  work  with  disruptions  and  recall.  Apparently  “relevance  to  the  goal  appears  to  be  a better  predictor 
of  an  actions  long  term  recall”  (p.  171-172).  Finally,  Henderson's  (1990)  study  could  be  viewed  as  an  example  of 
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relevance  in  that  the  students’  cultural  value  of  money  has  made  money  useful.  Hence,  by  involving  money  into  the 
design  of  instruction,  Henderson  has  introduced  a factor  of  relevance  into  the  subject  matter  of  accounting. 

Questions  regarding  the  utility  of  a topic  may  be  answered  quite  differently  than  questions  regarding  its 
interest.  In  fact,  in  work  related  trainings,  participants  may  hold  the  instruction  valuable  in  utility  and  its  importance 
to  their  job;  yet,  find  it  of  little  interest.  High  ratings  of  utility  and  low  ratings  of  interest  could  both  be  presented 
under  the  larger  term  of  relevance  and  give  very  different  assessment  of  the  “relevance”  of  the  instruction.  Studies 
that  equate  relevance  with  utility  or  usability  are  also  prevalent  (Quilter  & Harper,  1988;  Mahoney  & Lyday,  1984; 
Cennamo  & Braunlich,  1996;  Roser,  1990;  Sass,  1989;  and  Bainbridge-Frymier  & Shulman,  1995).  Quilter  and 
Harper  (1988),  for  example,  found  that  a lack  of  utility  was  the  main  deterrents  to  success  in  mathematics  of  152 
college  degreed  adults.  After  conducting  both  surveys  and  interviews  the  highest  scoring  variable  was  "relevance  of 
mathematics”  thus  supporting  the  importance  of  relevance  in  the  instructional  process. 

In  an  examination  of  computer-based  training,  Mahoney  and  Lyday  (1984)  further  support  the  importance 
of  relevance.  When  designing  CBT  for  employees  within  an  organization,  the  CBT  should  focus  on  the  learner's 
organization,  as  well  as  on  the  learner.  This  dual  focus  builds  in  factors  of  relevance  and  usefulness  so  that  a CBT 
learner  can  immediately  see  how  this  training  relates  to  his  or  her  job.  Small  and  Gluck  (1994)  used  a magnitude 
scaling  approach  to  determine  the  relationship  between  35  related  terms  and  the  four  ARCS  components.  Results 
showed  that  the  closest  relationship  is  between  relevance  and  the  related  term  meaningfulness.  This  result  supports 
the  role  of  value  in  determining  relevance. 

Relevance  and  Prior  Knowledge 

Instructional  designers  routinely  assess  learners’  prior  knowledge  of  an  instructional  topic  and  link  new 
content  to  it,  and  prior  experiences  have  been  previously  linked  to  perceptions  of  relevance.  The  relationship 
between  these  two  learner  characteristics,  however,  is  not  clear.  Smith  and  Ragan  (1999)  urge  designers  to  draw  on 
learners’  previous  knowledge  to  facilitate  the  connections  between  learner  prior  knowledge  and  the  instructional 
content.  One  way  to  clarify  the  nature  of  relevance  in  instruction  and  learning  may  be  to  draw  on  its  link  to 
cognition  and  prior  knowledge.  If  the  definition  of  relevance  were  expanded  to  include  to  emphasize  one’s  prior 
knowledge,  then  activation  of  that  knowledge  may  promote  perceptions  of  relevance. 

The  question  of  the  role  of  the  activation  of  prior  knowledge  in  learning  is  also  unanswered.  Can  activation 
facilitate  perceived  relevance  and  expand  motivation,  and  as  a result,  increase  learning?  Does  awareness  of  the  link 
between  prior  knowledge  and  relevance  have  any  impact  on  motivation  and  learning?  Researchers  have  looked  at 
cognition  and  learning,  motivation  and  learning,  but  have  not  systematically  studied  the  relationships  of  cognition, 
motivation  and  learning. 

From  a definition  standpoint,  it  would  no  longer  matter  if  instruction  is  perceived  as  useful  or  interesting. 
What  would  matter  would  only  be  the  learner’s  related  prior  knowledge.  This  would  determine  relevancy.  Since 
many  have  affirmed  that  perceptions  of  relevance  increase  motivation,  one  could  logically  posit  that  the  activation  of 
prior  knowledge  would  also  promote  motivation,  and  subsequently  learning  itself.  This  was  the  problem  addressed 
by  this  study. 

Conceptual  Model 

Figure  1 shows  the  conceptual  model  that  guided  this  study.  It  hypothesizes  an  explanation  of  how 
previous  experience  with  an  instructional  content  combined  with  activation  of  this  previous  knowledge  effects 
relevance,  motivation  and  learning. 
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FIGURE  1.  Conceptual  Model 


This  model  was  designed  to  promote  further  exploration  of  some  of  the  processes  involved  in  learning. 
Beginning  with  the  presentation  of  content,  a memory  trace  may  be  activated  if  the  learner  has  had  previous 
experience  with  that  content.  The  question  is  whether  perceptions  of  relevance  and  motivation  are  extended  by 
activation  of  previous  experience  with  content,  thereby  increasing  learning.  In  the  case  of  novel  and  irrelevant 
content  areas,  the  question  becomes  can  activation  be  manipulated  to  achieve  similar  effects. 

This  model  also  included  two  metacognitive  processes.  The  learners’  awareness  of  the  link  brings  a new 
aspect  to  these  complex  relationships.  Connectionist  models  posit  that  new  information  is  linked  to  prior  knowledge 
with  similar  features.  This  mental  process  happens  swiftly  and  often  without  awareness.  This  model  suggests  that 
the  learners’  awareness  of  any  connection  or  link  between  their  previous  experience  and  the  instructional  content 
may  influence  perceptions  of  relevance,  motivation  and  learning.  It  also  suggests  learners  may  continually  activate 
knowledge  every  time  they  think  about  the  content  and/or  instructional  experience.  This  repeated  activation 
strengthens  the  connections  within  the  brain  and  thus  strengthens  learning  (McClelland,  1988).  This  model  proposes 
that  continued  activation  may  be  affected  by  one’s  previous  experience  with  the  topic,  and  an  individual’s  awareness 
how  the  topic  connects  to  his  or  her  prior  knowledge.  These  may  in  turn  lead  to  an  increase  in  perceptions  of 
relevance  and  the  other  ARCS  variables  of  attention,  confidence,  and  satisfaction.  Their  role  in  a this  model  and 
their  influence  on  motivation  is  unique. 


Research  Design 

Population 

The  population  for  this  study  was  142  volunteer  Instructional  Technology  graduate  students.  These  students 
range  in  age  from  mid-twenties  to  over  60,  and  are  typically  employed  full-time  while  pursuing  their  degrees  part- 
time.  They  have  a wide  range  of  undergraduate  degrees  and  hold  jobs  in  fields  such  as  elementary  and  secondary 
education,  higher  education,  nursing,  computer  technology,  automotive  production,  industry  training,  and  non-profit 
work. 

Research  Design 

This  study  used  a variation  of  the  Solomon  Four  experimental  design.  The  independent  variables  were 
previous  experience  with  the  instructional  content,  perceived  relevance  (interest,  value  & use)  of  the  content,  and 
perceived  motivation.  The  dependent  variable  was  learning  as  determined  by  completion  of  a test  of  knowledge 
acquired.  The  four  groups  manipulated  in  this  study  were: 

1.  familiar  content/  with  activation  of  prior  knowledge  strategy; 

2.  familiar  content  / with  no  activation  of  prior  knowledge  strategy; 

3.  unfamiliar  content  / with  activation  of  prior  knowledge  strategy  and 

4.  unfamiliar  content/  with  no  activation  of  prior  knowledge  strategy. 

Table  1 presents  the  research  design  guiding  this  study. 
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Table  1.  Study  Design 


Ri  Pe/r//m 

Of 

2 week  delay 

Ri 

Act.  of  Of 

Tr/m 

Or 

R2 

No  Act.  of  Of 

Ox 

Tr/m 

R2  Pe/r/m 

Ou 

2 week  delay 

R3 

Act.  of  Ou 

Tr/m 

Or 

R4 

No  Act.  of  Ou 

Or 

Tr/m 

R Random  assignment  into  treatment  groups 

Pr/mr  Pre-  measure  of  previous  experience,  perceived  relevance  of  content  and 
motivation  to  learn 

Op  Presentation  of  familiar  content  with  which  subjects  will  have  had  some  prior 

experience 

Ou  Presentation  of  unfamiliar  content  with  which  subjects  will  not  have  had  prior 

experience 

Tr/m  Post-  measure  of  processing,  perceived  relevance  of  content  and  motivation  to  learn 
Or  Test  of  knowledge  acquisition 

Strategies  to  Control  Prior  Knowledge 

The  most  obvious  way  to  control  the  amount  and  level  of  prior  knowledge  is  to  control  the  topic.  Subjects 
are  not  likely  to  have  much  previous  knowledge  if  the  topic  is  novel  or  fictitious.  Hence  there  will  be  far  fewer  units 
to  activate  and  connect  to  the  new  information.  (See  Machiels  -Bongaerts,  Schmidt  and  Boshuizen,  1995.)  One  can 
argue  that  the  topics  selected  in  this  study  represent  varying  levels  of  relevance.  In  the  first  group,  the  topic  not  only 
activated  prior  knowledge,  but  also  had  interest  and  value  to  the  subjects.  The  second  group,  which  had  the  more 
obscure  topic  lacked  interest  and  perceived  value  as  well  as  extensive  prior  knowledge.  The  use  of  instructional 
topics  with  differing  levels  of  prior  knowledge  will  be  used  in  this  study  to  manipulate  both  relevance  and  prior 
knowledge  activation. 

A list  of  potential  topics  was  compiled  and  piloted  with  a group  of  12  Masters  students.  They  were  asked 
to  rate  each  topic  on  a Likert  sale  corresponding  to  the  amount  of  previous  experience  and  knowledge  they  had  on  a 
topic  as  well  as  their  interest,  use  and  perceived  relevance  of  that  topic.  The  topic,  migration  patterns  of  Canadian 
moose,  was  identified  as  the  least  interesting  and  least  useful.  It  also  had  the  lowest  rating  of  previous  experience 
and  prior  knowledge  (average  rating  4.75  for  prior  experience  and  4.08  for  relevance).  This  sample  also  indicated 
that  the  topic  of  most  interest  and  relevance  was  “APA  formatting  Guidelines”  (average  2.4  for  prior  knowledge  and 
experience  and  1.8  for  relevance).  Therefore,  these  two  instructional  topics  were  selected  to  control  for  previous 
knowledge  and  relevance. 

Treatment  and  Assignment 

Subjects  were  randomly  assigned  to  one  of  two  groups:  familiar/relevant  content  (APA  guidelines)  vs. 
unfamiliar/irrelevant  content  (migration  of  moose).  Subjects  completed  a “pre -instruction”  questionnaire  to  identify 
their  knowledge  of  the  topic  and  perceptions  of  attention,  relevance,  confidence  and  satisfaction  for  the  topic.  This 
questionnaire  was  based  on  the  instrument  created  by  Keller  & Subhiyah  (cited  in  Bohlin,  Milheim,  & Viechnicki, 
1993-94). 

Subjects  in  the  unfamiliar/irrelevant  content  group  received  instruction  on  a topic  with  which  they  were 
predicted  to  have  no  prior  knowledge.  The  instruction  involved  a 15  -minute  presentation  on  the  migration  patterns 
of  a herd  of  Canadian  moose.  Subjects  in  the  familiar/relevant  content  group  received  a 15-minute  presentation  on 
how  to  cite  electronic  references  in  APA  format.  A professional  trainer  delivered  the  instruction  and  used  a 
PowerPoint  slide  presentation  to  guide  the  delivery. 

After  a two-week  delay,  subjects  were  randomly  assigned  to  two  sub  groups  (those  who  would  receive 
specific  activation  strategies  and  those  who  would  not).  Subjects  comp  leted  a post-instruction  questionnaire  and  a 
test  to  measure  their  learning  of  the  content. 

Data  Analysis 

Factorial  ANOVAs  were  used  to  identify  differences  in  the  dependent  variable  and  the  treatment  groups 
and  differences  related  to  perception  ofrelevance  and  motivation  and  learning.  MANOVAs  were  used  to  identify 
any  relationships  between  the  activation  treatment  and  subjects’  perceptions.  Finally  to  test  the  proposed  conceptual 
model,  a path  analyses  was  also  completed.  This  analysis  included  variables  on  subjects’  awareness  of 
metacognitive  processes  and  continued  activation  in  addition  to  previous  experience,  perceptions  all  to  predict 
learning  as  the  desired  outcome.  A secondary  analysis  was  also  completed  to  explain  selected  unanticipated 
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outcomes  pertaining  to  influences  on  learning,  and  the  role  of  the  individual  ARCS  variables  and  the  model  as  a 
whole. 


Results 

Effects  of  Perceptions  of  Relevance  and  Perceptions  of  Motivation  on  Learning 

This  study  examined  the  differences  in  learning  between  groups  with  varying  perceptions  of  topic 
relevance  and  varying  levels  of  motivation.  It  addressed  differences  in  learning  between  high  and  low  perceptions  of 
relevance  and  high  and  low  perceptions  of  motivation.  A 2 x 2 factorial  ANOVA  was  used  with  learning  as  the 
dependent  variable.  Perceptions  of  relevance  were  taken  from  the  questionnaire  and  rated  as  low  or  high. 
Perceptions  of  motivation  were  likewise  used.  Table  2 presents  the  data  from  this  ANOVA. 


Table  2 Differences  in  Learning  by  Perceptions  of  Relevance  and  Motivation 


Source  of  Variance 

Sum  of 

Mean 

Squares 

df 

Square 

F 

Sig 

Explained  Variance 

Perceptions  of  Motivation 

.0001 

1 

.0001 

.003 

.956 

Perceptions  of  Relevance 

.0000 

1 

.0000 

.000 

.995 

Perceptions  of  Motivation  x 
Perceptions  of  Relevance 

.017 

1 

.017 

.491 

.485 

Unexplained  Variance 

4.23 

120 

.035 

Total  Variance 

4.25 

123 

The  comparison  of  learning  by  perceptions  of  motivation  produced  an  F ratio  of  .003  which  was  not  statically 
significant  at  a .05  alpha  level.  When  perceptions  of  relevance  were  compared,  the  F ratio  of  .000  was  also  not 
significant.  The  interaction  effect  of  motivation  and  relevance  produced  an  F ratio  of  .491  for  total  learning  which 
was  not  statically  significant.  Descriptive  statistics  were  obtained  to  examine  the  results  between  groups.  These 
descriptive  statistics  are  presented  in  Table  3. 

Table  3 Descriptive  Statistics  for  Learning  by  Perceptions  of  Relevance  and  Motivation 


Independent  Variable  N Mean  SD 


Perceptions  of  Relevance 
High 
Low 

63 

61 

.59 

.59 

.19 

.19 

Perceptions  of  Motivation  (ACS) 

High 

60 

.58 

.18 

Low 

64 

.59 

.19 

Relevance  x Motivation 

High  Relevance  x High  Motivation 

57 

.58 

.18 

High  Relevance  x Low  Motivation 

6 

.63 

.21 

Low  Relevance  x Low  Motivation 

58 

.59 

.19 

Low  Relevance  x High  motivation 

3 

.63 

.13 

There  was  no  difference  in  the  mean  scores  of  groups  reporting  high  levels  of  relevance  (m=.  59,  SD=.  19) 
vs.  those  who  reported  low  perceptions  of  relevance  (m=.  59,  SD=.  19).  Likewise  there  was  no  difference  between 
groups  reporting  high  levels  of  motivation  (m=.  58,  sd-.  18)  and  groups  reporting  low  levels  of  motivation  (m=.  59, 
SD=.  19).  Finally,  the  interaction  between  perceptions  of  relevance  and  perceptions  of  motivation  did  not  produce 
statistically  significant  differences.  These  findings  suggest  that  neither  perceptions  of  relevance  nor  perceptions  of 
motivation  were  influential  in  determining  learning  in  this  study. 

Effects  of  Activation  on  Perceptions  of  Relevance,  Perceptions  of  Motivation  and  Learning 

The  effect  of  activation  strategies  was  also  questioned  in  this  study.  This  analysis  examined  differences  in 
perceptions  of  motivation,  perceptions  of  relevance  and  learning  when  memory  is  activated.  This  analysis  also 
included  perceptions  of  motivation  and  relevance  and  learning  and  but  this  time  examined  them  in  relation  to  the 
activation  treatment.  To  determine  if  there  was  a difference  in  perceptions  of  relevance  and  motivation  and  in 
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learning,  a one-way  multiple  analysis  of  covariance  (MANCOVA)  was  used.  The  independent  variable  in  this 
analysis  was  memory  activation,  with  post  questionnaire  scores  on  perceptions  of  relevance  and  motivation  were 
used  as  covariates.  Table  4 presents  the  results  of  the  MANCOVA. 


Table  4 Multiple  Analysis  of  Covariance  - Relevance,  Motivation  and  Learning  by  Memory  Activation 

Hotelling’s  Trace F Ratio Df Sig  of  F 

~I3  5^03  3,  118  ^03* 

The  MANCOVA  resulted  in  a Hotelling’s  trace  of . 13.  The  F ratio  of  5.03  was  statically  significant  at  an  alpha 
level  of  .05  with  3 and  118  degrees  of  freedom.  The  two  covariates,  perceptions  of  relevance  F (3,  118)  = 38.45, 
p<.001  and  perceptions  of  motivation  F (3,  118)  = 30.35,  p<001.  To  determine  which  of  the  dependent  variables 
was  contributing  to  the  statically  significant  results,  the  univariate  F test  was  completed.  The  results  of  this  analysis 
are  presented  in  Table  5. 

Table  5 Univariate  F tests  - Relevance,  Motivation  and  Learning  by  Activation 

N M Std.  D df  F Sig. 


Perceptions  of  Relevance 
Nonactivation  Group 
Activation  Group 

59 

65 

13.42 

13.43 

6.90 

7.00 

1,  120 

3.83 

.053 

Perceptions  of  Motivation 

Nonactivation  Group 

59 

28.19 

12.46 

1,  120 

5.69 

.019* 

Activation  Group 

65 

28.86 

12.09 

Learning 

Nonactivation  Group 

59 

.54 

.18 

1,  120 

8.92 

.003* 

Activation  Group 

65 

.63 

.18 

Two  of  the  three  measures,  perceptions  of  motivation  and  learning  differed  significantly  between  students  who 
received  the  activation  strategies  and  those  who  did  not.  The  difference  between  the  two  groups  on  perceptions  of 
relevance  did  not  differ  significantly. 

The  F ratio  of  5.69  resulting  from  perceptions  of  motivation  was  significant  at  an  alpha  level  of  .05  with  1 
and  120  degrees  of  freedom.  An  examination  of  the  mean  scores  showed  that  students  who  received  the  memory 
activation  strategy  (m=28.86,  ad=12.09)  had  significantly  higher  motivation  than  those  who  did  not  receive  the 
activation  treatment  (m=28.19,  SD=12.46) 

A comparison  of  the  learning  scores  between  the  groups  resulted  in  an  F ratio  of  8.92,  which  was  statically 
significant  at  an  alpha  of  .05  with  1 and  120  degrees  of  freedom.  This  result  indicates  that  students  who  were 
assigned  to  the  memory  activation  group  (m  = .63,  as=18)  had  significantly  higher  scores  than  students  who  were 
not  in  the  activation  group  (m=.54,  SD  = 18).  These  results  demonstrate  the  positive  impact  of  the  activation 
treatment  on  learning. 

Test  of  Conceptual  Model 

This  study  addressed  what  the  relationships  among  learn  ing  and  previous  experience  with  content, 
activation  of  memory  trace,  relevance  and  motivation.  The  hypothesized  model  presented  previously  in  Figure  Iwas 
tested  using  path  analysis  techniques.  Table  6 presents  these  findings  and  Figure  2 presents  the  resulting  model. 
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Table  6 Factors  Predicting  Learning 


Dependent 

Variable 


Causal  Variables  b SE  B 6 T Sig.  T p^th 


Learning 

Constant 

.525 

.077 

Motivation  Factor  (ACS) 

.054 

.026 

.288 

2.09 

.039* 

.ill 

Perceptions  of  Relevance 

-.015 

.005 

-.545 

-2.91 

.004* 

.064 

Awareness  of  Link 

.036 

.020 

.301 

1.83 

.070 

.136 

Memory  Activation 
R Square  = .138 

.080 

.032 

.217 

2.51 

.013* 

.063 

Endogenous 

Variable 

Causal  Variables 

b 

SEB 

6 

T 

Sig.  T 

^2  tor 
path 

Motivation 

Constant 

-1.84 

.224 

Factor  (ACS) 

Perceptions  of  Relevance 

.076 

.016 

.526 

4.66 

.000* 

.632 

Continued  Activation 

.036 

.017 

.156 

2.17 

.032* 

.564 

Awareness  of  Link 
R Square  = .632 

.155 

.068 

.240 

2.28 

.025* 

.537 

Perceptions 

Constant 

.332 

1.26 

of  Relevance 

Continued  Activation 

.158 

.092 

.099 

.1.72 

.089 

.756 

Awareness  of  Link 

3.15 

.250 

.706 

12.63 

.000* 

.750 

Previous  Experience 
R Square  = .756 

.261 

.087 

.174 

2.99 

.003* 

.357 

Continued 

Constant 

2.88 

1.22 

Activation 

Awareness  of  Link 

.823 

.235 

.296 

3.50 

.001* 

.381 

Previous  Experience 
R Square  = .388 

.380 

.079 

.408 

4.84 

.000* 

.317 

Awareness  of 

Constant 

1.22 

.458 

Link  to  Prior 

Previous  Experience 

.175 

.026 

.523 

6.76 

.000* 

..269 

Knowledge 

R Square  = .276 

Alpha  = .05  N = 124 


FIGURE  2.  Resulting  Model 
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Secondary  Analyses 

The  results  of  the  above  analyses  raised  some  questions.  While  the  literature  predicted  differences  in 
learning  based  on  previous  experience,  prior  knowledge  and  perceptions  of  relevance  and  motivation,  these  results 
show  that  learning  was  only  affected  by  the  activation  strategy.  However,  learning  was  measured  by  correct 
responses  on  an  achievement  test.  A more  extensive  examination  of  this  learning  variable  was  completed  in  an 
effort  to  understand  the  unanticipated  lack  of  influence  of  prior  experience  and  motivation  variables.  The  mean 
achievement  score  was  59%  with  a standard  deviation  of  0.19.  The  majority  of  subjects  scored  in  the  range  of  55% 
to  78%  correct.  These  scores  represent  a lower  than  average  test  performance  regardless  of  instructional  topic.  The 
lower  scores  could  help  explain  the  lack  of  significant  effects  from  previous  experience  and  motivational  variables. 
The  results  pertaining  to  the  influence  of  motivation  variables  were  also  unexpected.  To  examine  the  relationship 
among  the  motivation  variables  and  learning,  a correlation  matrix  was  created.  Table  7 presents  the  results. 


Table  7 Correlations  for  Motivation  Variables  and  Learning 

Learning  Perceptions  of  Perceptions  of  Perceptions  of  Perceptions  of 
Relevance  Satisfaction  Attention  Confidence 


Learning 

Perceptions  of 
Relevance 

Correlation 

-.033 

Significance 

.714 

Perceptions  of 
Satisfaction 

Correlation 

.113 

.732* 

Significance 

.121 

.000 

Perceptions  of 
Attention 

Correlation 

.105 

.639* 

.850* 

Significance 

.245 

.000 

.000 

Perceptions  of 
Confidence 

Correlation 

.199 

.625* 

.549* 

.430* 

Significance 

.189 

.000 

.000 

.000 

N=  124 


These  data  indicate  no  significant  correlation  between  variables  of  motivation  and  learning.  In  addition,  the 
correlation  matrix  illustrates  how  highly  correlated  the  ARCS  components  are.  They  act  as  suppressors  to  the  entry 
of  other  independent  variables  into  a regression.  It  is  interesting  to  note  negative  correlation  between  perceptions  of 
relevance  and  learning.  This  correlation  indicates  lower  ratings  for  relevance  actually  resulted  in  higher  scores  on 
the  final  test. 

Manipulation  of  ARCS 

Secondary  analyses  were  completed  due  to  the  manipulation  of  ARCS.  Keller’s  ARCS  model  of 
motivation  (1987a)  suggests  that  motivation  is  made  up  of  an  attention  variable,  a relevance  variable,  a confidence 
variable  and  a satisfaction  variable.  How  these  variables  are  treated  can  vary.  For  example,  this  study  combined 
them  in  some  cases  to  create  one  variable  for  overall  motivation.  In  other  situations,  they  were  each  treated 
separately  as  four  distinct  variables.  Yet,  this  study  was  particularly  interested  in  the  role  of  relevance  as  it  relates  to 
prior  knowledge,  the  activation  strategy,  the  other  ARCS  variables  of  attention,  confidence  and  satisfaction  and 
finally  learning.  As  such,  the  variable  of  relevance  was  kept  separate  for  much  of  the  study.  However,  this  created  a 
statistical  problem.  If  relevance  was  separated,  any  analysis  relating  it  to  overall  motivation  could  not  use  ARCS 
(all  four  variables)  as  a referent  due  to  collinearity.  Relevance  could  not  be  used  in  both  its  own  variable  and  in  an 
overall  motivation  variable.  A series  of  secondary  analyses  were  completed  to  understand  the  dynamic  among  the 
ARCS  variables. 

First  a reliability  coefficient  was  calculated  for  items  6 through  15  on  the  instruments  --  those  measuring 
perceptions  of  attention,  relevance,  confidence  and  satisfaction.  The  resulting  coefficient  was  .923  with  an  N of  124 
signifying  internal  consistency  and  good  reliability  for  the  instruments  as  a measure  of  perceptions  of  the  ARCS 
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variables.  However,  when  a factor  analysis  was  competed  for  these  items , they  loaded  as  two  separate  factors  but 
with  no  logical  explanation  for  the  split.  Table  8 presents  to  results  of  this  analysis. 

Table  8 Factor  Analysis  of  Questionnaire  Items  Related  to  Perceptions  of  Motivation 


Questionnaire  Item 

Factor  One 

Factor  Two 

Item  6 - Relevance 

.816* 

.353 

Item  7 - Relevance 

.943* 

.150 

Item  8 - Relevance 

.895* 

.286 

Item  9 - Awareness  of  Link 

Not  a measure 

of  motivation 

Item  10  - Relevance 

.861* 

.400 

Item  1 1 - Relevance 

.219 

.913* 

Item  12  - Attention 

.379 

.872* 

Item  1 3 - Attention 

.257 

.905* 

Item  14  - Confidence 

.630* 

.339 

Item  15  - Satisfaction 

.397 

.823* 

These  data  indicate  that  Factor  One  consists  of  items  6,  7,  8,  10,  and  14  (four  items  related  to  relevance  and  one  item 
related  to  confidence).  Factor  Two  consists  of  items  1 1,  12,  13,  and  15  (one  item  related  to  relevance,  two  items 
related  to  attention  and  one  item  related  to  satisfaction). 

An  additional  factor  analysis  was  completed  regarding  the  variables  of  perceptions  of  attention,  relevance, 
confidence  and  satisfaction.  When  these  four  individual  variables  were  loaded  they  loaded  as  one  variable 
representing  overall  motivation.  Table  9 presents  the  results  of  this  analysis. 


Table  9 Factor  Analysis  of  Perception  of  Motivation  Variables 


Factor  One 

Perception  of  Attention 

.888 

Perceptions  of  Relevance 

.871 

Perceptions  of  Confidence 

.640 

Perceptions  of  Satisfaction 

.938 

These  data  support  the  concept  of  the  ARCS  model  of  motivation  as  one  unified  variable;  however, 
relevance  was  of  particular  interest  in  this  study.  It  was  a manipulated  variable  through  selection  of  instructional 
topic  and  use  of  the  activation  strategy.  As  such  relevance  was  statistically  removed  from  ARCS  and  treated  as  one 
factor  leaving  attention,  confidence  and  satisfaction  as  a second  factor  representing  motivation.  Table  10  presents 
the  resulting  factor  coefficients. 


Table  10  Factor  Manipulation  of  ARCS  Variables 


Questionnaire  Item 

Factor  One  - 
Perceptions  of 
Relevance 

Factor  Two  - 
Perceptions  of 
Motivation  (ACS) 

Relevance 

.816 

Attention 

.888 

Confidence 

.640 

Satisfaction 

.938 

Item  10  - Relevance 

.861 

.400 

These  factor  coefficients  were  subsequently  used  in  the  regressions  for  the  path  analysis  presented  in  Figure  2. 


Discussion 

This  study  examined  the  degree  to  which  prior  knowledge,  activation  of  that  knowledge,  specific 
metacognitive  variables,  and  perceptions  of  relevance  and  motivation  influence  learning.  It  emphasized  the  role 
activation,  relevance,  and  motivation  play  with  respect  to  learning  when  they  are  combined  and  manipulated.  By 
further  understanding  these  relationships,  instructional  designers  and  educators  may  improve  their  instructional 
strategies,  build  a basis  for  incorporating  metacognitive  strategies  into  instruction  and  enhance  motivation  itself 
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This  study  sought  to  apply  the  concept  of  activation  to  instructional  design.  It  tested  the  utility  of  building  on 
learners’  previous  knowledge  to  facilitate  the  connections  with  new  instructional  content.  This  study  examined 
activation  of  prior  knowledge  as  a means  of  facilitating  relevance  and  motivation.  Such  a relationship  would 
provide  support  for  existing  design  practices  such  as  content  sequencing,  use  of  advanced  organizers,  transfer 
strategies,  and  applied  the  recent  tenets  of  connectionism  to  them.  A relationship  between  the  cognitive  process  of 
activation  and  relevance  would  provide  additional  theoretical  support  for  relevance  as  an  important  element  in 
learning.  Finally,  the  complex  relationships  between  prior  knowledge,  motivation  and  their  effects  on  learning 
might  be  more  fully  understood. 

The  second  purpose  of  this  study  was  to  add  to  the  understanding  of  the  ARCS  model  of  motivation 
(Keller,  1984,  1987a),  especially  the  relevance  component.  Relevance  has  less  research  support  and  may  be  related 
to  connectionism  if  relevant  information  can  act  as  an  activation  cue.  A main  tenet  of  ARCS  is  that  it  influences 
learning  (Keller,  1987a,  1987b).  The  literature  is  rich  with  data  supporting  the  role  of  attention,  satisfaction  and 
confidence  in  learning,  and  some  data  exist  that  lend  support  to  relevance’s  influence  on  learning.  Yet,  research  on 
ARCS  as  a unified  model  of  motivation  is  needed. 

The  resulting  model  seen  in  Figure  2 suggests  the  role  of  prior  knowledge,  activation  and  motivation  in 
learning.  These  resulting  relationships  are  consistent  with  the  literature.  Activation  of  prior  knowledge  clearly  has 
an  impact  on  recall  and  subsequent  learning  (Anderson,  1981;  Carr  and  Thompson,  1996;  Madler,  in  Kihlstrom, 
1999;  Diamond  and  Rozin,  in  Kihlstrom,  1999;  Burgess  and  Hitch,  1992;  McClelland  and  Rumelhart,  1988). 
Motivation  also  influences  learning  (Fleming,  1987;  Dweck,  1986;  Dweck  & Leggett,  1988;  Mahoney  & Lyday, 
1984;  Small  and  Dodge,  1996;  Schiefele,  1991). 

This  model  suggests  the  role  of  prior  knowledge,  activation  and  motivation  in  learning.  These  resulting 
relationships  are  consistent  with  the  literature.  Activation  of  prior  knowledge  clearly  has  an  impact  on  recall  and 
subsequent  learning  (Anderson,  1981;  Carr  and  Thompson,  1996;  Madler,  in  Kihlstrom,  1999;  Diamond  and  Rozin, 
in  Kihlstrom,  1999;  Burgess  and  Hitch,  1992;  McClelland  and  Rumelhart,  1998).  Motivation  also  influences 
learning  (Fleming,  1987;  Dweck,  1986;  Dweck  & Leggett,  1988;  Mahoney  & Lyday,  1984;  Small  and  Dodge,  1996; 
Schiefele,  1991). 

The  role  of  activation  in  learning  is  supported  by  the  previous  research.  Activation  of  prior  knowledge 
helps  connect  new  information  to  prior  knowledge.  Activation  of  prior  knowledge  affected  subjects’  learning  in  both 
instructional  topics  in  this  study.  Furthermore,  instructional  strategies  to  facilitate  activation  improved  subjects’ 
recall  of  this  information  for  the  final  test. 

This  study  hypothesized  that  activation  of  prior  knowledge  would  influence  perceptions  of  relevance.  This 
belief  prompted  the  isolation  of  relevance  from  the  other  ARCS  components,  but  the  results  did  not  support  the 
hypothesized  influence  of  activation  on  perceptions  of  relevance.  This  finding  is  puzzling,  but  a possible 
explanation  lies  in  the  collinearity  of  the  ARCS  components.  Given  the  high  correlation  of  each  component,  when 
relevance  was  removed,  the  remaining  three  elements  (ACS)  account  for  so  much  of  the  variability  that  little  is  left 
for  perceptions  of  relevance  to  affect. 

The  data,  however,  do  support  the  activation  strategy’s  effect  on  learning  since  subjects  who  received  the 
activation  intervention  performed  significantly  higher  on  the  final  test.  These  findings  lend  additional  support  to  the 
connectionism  literature  and  may  explain  the  information  retrieval  process.  They  support  the  idea  that  “retrieval  of 
an  item  from  memory  consists  of  activating  its  internal  representations,  that  activation  of  a node  spreads  through  the 
links  to  other  nodes  in  the  network”  (McNamara,  1999,  p.  118).  In  this  study,  presentation  of  a cue  to  subjects  in  the 
activation  group  activated  their  representations  of  “Moose”  and  “APA  Guidelines.”  This  activation  may  have  also 
impacted  related  knowledge  and  strengthened  information  retrieval  on  the  test. 

Activation  of  prior  knowledge  predicts  learning,  but  it  also  important  in  other  ways.  The  results  of  the 
experiment  suggest  that  activation  influenced  perceptions  of  motivation.  Activation  also  appears  to  influence 
changes  in  subjects’  perceptions  of  relevance,  confidence  and  motivation  from  pre-to  post  instruction.  Activation  of 
prior  knowledge  may  a powerful  tool  for  instructional  designers 

This  study  attempted  to  build  on  the  work  of  Keller  (1984,  1987a,  1987c)  and  his  ARCS  model  of 
motivation.  The  results  of  this  study  greatly  aid  in  the  understanding  of  ARCS.  Examining  ARCS  in  terms  of  its 
components  has  some  appeal.  Bohlin,  Milheim,  & Viechnicki,  (1993),  Small  & Gluck  (1994)  and  Means  (1997) 
took  this  approach.  However,  the  findings  of  this  study  provide  little  justification  for  isolating  the  ARCS  variables. 
These  data  do  not  show  that  the  components  relate  to  learning.  What  they  do  show  is  that  the  ARCS  as  a whole 
does  influence  learning.  The  support  for  ARCS  as  a unified  model  of  motivation  is  a significant  contribution  to  the 
field  since  there  is  little  empirical  research  that  does  so. 

Much  of  the  difficulty  of  studying  ARCS  lies  in  determining  appropriate  measures  of  motivation.  This  study 
used  Bohlin,  Milheim,  & Viechnicki’ s ( 1993)  variation  of  Keller’s  Course  Interest  Survey.  The  instrument 
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contained  question  items  pertaining  to  each  ARCS  component,  but  the  factor  analysis  here  found  that  these  items  did 
not  group  as  intended.  Some  questions  involving  perceptions  of  relevance  combined  with  questions  of  attention, 
while  other  relevance  items  joined  items  that  measured  satisfaction.  For  example,  questionnaire  item  1 1 asked 
subjects  about  their  interest  in  the  instructional  topics.  Interest  is  usually  included  in  discussions  of  relevance; 
however,  interest  can  also  be  considered  a feature  of  attention.  The  results  of  this  study  suggest  it  may  not  be 
necessary  to  measure  individual  ARCS  components  since  ARCS  appears  to  be  one  unified  model  of  motivation. 
However,  if  there  is  a need  to  study  individual  parts  of  ARCS,  preexisting  measures  may  need  to  be  reexamined. 

How  the  variables  are  defined  is  critical.  The  influence  of  prior  experience  on  perceptions  of  relevance  supports  a 
call  for  a wider  definition  of  relevance  - one  based  on  prior  knowledge  and  experience  and  less  on  subjective 
elements  like  interest  and  utility. 

This  study  confirms  previous  research  that  supports  the  use  of  motivation  design  using  the  Keller  model 
(1987  b,  1987c,  1999)  that  emphasizes  attention  getting  strategies,  relevance  building  strategies,  confidence  building 
strategies  and  satisfaction  building  strategies.  While  many  designers  are  aware  of  these  strategies,  few  actually  use 
them  in  their  designs  (Shellnut,  1999).  Given  both  the  direct  and  indirect  influence  of  motivation  variables  on 
learning,  it  seems  that  motivational  design  should  be  adopted  as  standard  practice  when  deigning  instruction. 

This  study  had  conflicting  results  with  regards  to  the  roles  of  activation  of  prior  knowledge  and  the  influence  of 
previous  experience  in  learning.  For  example,  there  were  no  differences  in  learning  between  those  who  reported 
previous  experience  with  the  topic  (the  APA  group)  and  those  who  did  not  (the  Moose  group).  This  might  be 
explained  in  two  ways.  First,  while  the  topic  of  moose  migration  was  not  relevant,  it  was  potentially  interesting  or  at 
least  novel.  Hence,  subjects’  interests  may  have  dominated  and  counteracted  their  lack  of  previous  experience. 
Means’  study  (1997)  may  help  confirm  this  possible  explanation  of  the  results.  In  her  study,  she  attempted  to  isolate 
intrinsic  relevance  and  extrinsic  strategies  to  promote  relevance.  She  found  that  both  factors  enhanced  learners’ 
motivation.  The  internal  relevance  or  “interests”  compensated  for  any  external  motivational  design  strategies. 

Implications  for  Instructional  Design  Practice 

These  findings  offer  valuable  contributions  to  the  to  field  of  instructional  design  in  a number  of  ways.  One 
contribution  of  this  study  is  the  use  of  the  construct  of  activation  of  memory  traces  in  design  research  and  practice. 

In  a sense,  this  validates  the  common  instructional  design  practice  of  stimulating  prior  learning.  It  also  supports  the 
use  of  activation  as  an  instructional  strategy.  Activation  strategies,  in  this  study,  promoted  increases  in  perceptions 
of  confidence,  relevance,  and  overall  motivation  from  initial  measures  to  post-instructional  measures.  Activation 
strategies  also  directly  affected  learning.  The  findings  support  continued  research  and  use  of  strategies  to  activate 
prior  knowledge  in  learning  situations. 

A second  major  finding  of  this  study  is  the  support  for  ARCS  as  a unified  model  of  motivation.  This 
experiment  consistently  found  evidence  for  ARCS  as  a whole.  The  study  supports  Keller’s  work  by  continuing  to 
refine  measurement  of  ARCS  and  learner  motivation.  It  supports  ARCS’s  the  role  of  the  ARCS  variables  to  directly 
enhance  learning.  While  this  study  also  raise  questions  on  ARCS  and  its  manipulation,  the  data  clearly  indicate  the 
use  of  motivation  strategies  as  a way  to  promote  learning. 

The  conceptual  model  resulting  from  this  study  illustrates  that  the  activation  of  prior  knowledge, 
metacognitive  variables  of  self-awareness,  perceptions  of  relevance  and  motivation  all  have  varying  degrees  of 
influence  on  learning.  Designers  should  promote  strategies  that  activate  prior  knowledge,  encourage  learners’ 
awareness  of  knowledge  organization  and  increase  perceptions  of  motivation.  Some  might  ask,  “why  bother  to  study 
cognitive  process  factors  such  as  prior  knowledge  or  self-awareness  of  knowledge  organization?”.  By  studying  even 
the  simplest  cognitive  activities,  it  is  anticipated  that  we  can  eventually  understand  mental  processing  as  a whole  and 
those  processes  that  ultimately  shape  learning.  It  is  this  quest  to  facilitate  learning  that  drives  the  study  of 
instructional  design. 
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Abstract 

This  paper  presents  the  rationale  and  strategies  used  to  integrate  the  newly  mandated  State  Technology 
Application  Standards  into  the  secondary  education  program  at  the  University  of  Texas  Pan  American  without 
increasing  credit  hours  for  pre -service  teachers.  A review  of  the  literature  and  an  assessment  of  the  local 
infrastructure  facilitated  the  definition  of  an  action  plan  and  strategies  for  the  restructuring  process.  Implementation 
strategies  like  course  alignment  to  technology  standards,  incorporating  action  research  processes,  use  of  electronic 
learning  communities  and  electronic  portfolios  are  discussed. 

Introduction 

In  1997  in  a message  to  National  Council  of  Teacher  Education  (NCATE)  institutions  titled  Technology 
and  the  New  Professional  Teacher:  Preparing  for  the  2P‘  Century  Classroom  (1997)  Arthur  E.  Wise  stated:  “Two 
million  new  teachers  will  be  hired  over  the  next  decade.  Will  these  new  teachers  be  comfortable  and  skilled  in  using 
technology?  ...  As  technology  moves  from  the  periphery  to  the  center  in  P-12  schools,  so  must  it  move  from  the 
periphery  to  the  center  in  teacher  preparation.” 

Nearly  half  a decade  later  many  teacher  preparation  programs  still  grapple  with  the  logistics  of  creating  the 
conditions  by  which  candidates  can  teach  and  use  educational  technology  for  greater  student  learning  and 
achievement.  There  is  significant  research  that  supports  the  potential  of  technology  to  enhance  student  achievement 
and  teacher  learning,  but  only  if  it  is  used  appropriately  (Dede,  1998).  Teacher  preparation  programs  must  provide 
opportunities  for  preservice  teachers  to  acquire  the  necessary  foundation  and  skill  to  integrate  technology  into  the 
learning  process  in  a meaningful  way.  And  these  opportunities  must  be  provided  while  the  demand  to  produce 
teachers  fast  enough  to  meet  the  Nation’s  hiring  need  of  “2  million  teachers”  within  this  decade  remains  unchanged 
(Darling-Hammond,  2000).  Results  of  a survey  of  schools,  colleges  and  departments  of  education  in  the  United 
States  on  preparing  teachers  to  use  information  technology  were  published  by  the  International  Society  for 
Technology  in  Education  (ISTE)  in  the  Milken  Exchange  on  Education  Report  (1999).  The  report  stated  the 
following  obstacles  to  integrating  technology  into  teacher  preparation  programs: 

a)  One  third  of  the  institutions  surveyed  are  limited  by  deficiencies  in  their  IT  facilities, 

b)  Most  faculty  do  not  model  IT  skills  in  teaching, 

c)  Distance  education  affects  a small  proportion  of  students  in  teacher  training  institutions, 

d)  Most  teacher-preparation  programs  do  not  have  a technology  plan, 

e)  Most  students  do  not  routinely  use  technology  during  field  experience  and  do  not  work  with  mentors  or 
supervisors  who  can  advise  them  on  IT  use  and,  and  stand-alone  IT  courses  are  not  effective  in  helping 
teachers  to  integrate  IT  into  teaching. 

An  additional  challenge  to  change  preparation  programs  resides  in  the  perception  of  contemporary  interest 
groups  that  question  the  validity  of  teacher  education  programs  ' in  general,  including  their  basic  structure  and  even 
their  existence.  Current  teacher-preparation  programs  are  seen  by  some  as  a collection  of  disjointed  courses,  field 
experience  and  student  teaching — ^taught  or  overseen  by  people  who  have  little  ongoing  communication  with  each 
other  (Goodlad,  1990),  or  as  programs  with  inadequate  time,  fragmented,  uninspired  teaching  methods  and  a 
superficial  curriculum  (National  Commission  on  Teaching  and  America’s  Euture  (NCTAE,  1996).  In  the  face  of 
such  scrutiny,  teacher  educators  are  nonetheless  charged  with  infusing  current  research  in  cognition  and  instruction 
into  the  classroom  as  well  as  integrating  technology  into  all  coursework  without  an  increase  in  credit  hours. 

The  advent  of  new  State  professional  standards  that  include  technology  standards  for  all  teachers  stimulated 
the  formation  of  a technology  committee  to  research  approaches  to  technology  integration  for  the  teacher 
preparation  programs  at  the  University  of  Texas -Pan  American  (UTPA)  in  the  Department  of  Curriculum  and 
Instruction.  Eortunately,  the  committee  found  that  along  with  reports  that  highlight  disparate  preparation  of  pre- 
service  teachers  (NCTAE,  1996;  Betz,  & Mitchel,  1996),  there  is  growing  documentation  of  “best  practices”  that 
relate  how  future  teachers  can  develop  a professional  knowledge  base  and  skills  to  utilize  technology  and  change  the 
teaching-learning  cycle  of  classrooms  (Persichitte,  Caffarella  & Tharp,  1999;  Ropp,  1999;  Schrum,  1999).  Studies 
support  the  idea  of  infusing  technology  into  teaching  and  learning  as  a norm  not  Just  as  a “need”  for  teacher 
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preparation  programs.  Many  successful  efforts  to  integrate  technology  throughout  the  teacher  education  curriculum 
have  been  reported  (Schrum  & Dehoney,  1998;  Pope,  Hare,  & Howard,  2002;  Snider,  2002;  Vannatta  and 
O’Bannon,  2002).  As  Snider  (2002)  pointed  out  “curriculum- wide  integration  currently  represents  best  practice  as  it 
addresses  two  of  the  most  significant  barriers  to  the  institutionalization  of  educational  technology:  inservice  teacher 
resistance  and  faculty  inexperience’’  (Strudler  & Wetzel,  1999  as  cited  by  Snider,  2002).  A study  by  Moursund  & 
Bielefeldt  (1999)  reveals  that  most  faculty  did  not  feel  that  instructional  technology  training  was  adequate  or 
effectively  modeled  for  future  teachers  and  even  though  many  of  the  faculty  members  know  quite  a lot  about  using 
the  technologies  for  their  own  professional  work,  they  are  uncertain  how  to  integrate  it  into  their  classes. 

It  is  also  reported  that  pre -service  teachers  need  more  opportunities  to  see,  use  and  apply  IT  during  field 
experience  and  internships.  In  1999,  ISTE  reported  that  infusion  throughout  a teacher  education  program,  including 
field  experience,  rather  than  separate  training,  was  the  most  effective  method.  Despite  these  findings,  few  education 
teacher  preparation  programs  require  the  inclusion  of  technology  in  field  experience.  However,  the  effective  use  of 
technology  during  field  experience  practicum  has  been  documented  by  Powell  and  Lord  (1998)  and  Anderson  and 
Petch-Hogan  (2001).  These  authors  concur  that  early  exposure  to  technology  during  field  experience  may  have  a 
positive  impact  on  future  classroom  practices.  Anderson  and  Petch-Hogan  (2001)  recommend  increased 
opportunities  for  pre-service  students  to  “take’’  technology  into  their  fieldwork  experience. 

The  ongoing  trend  toward  integrating  technology  into  teacher  preparation  programs  is  now  well 
documented  (e.g.,  Schrum  & Dehoney,  1998;  Persichitte,  Cafferella  & Tharp,  1999;  Willis  & Tucker,  2001; 

Vannatta  & O’Bannon,  2002).  However,  there  is  a continued  gap  between  the  advocacy  of  best  practices  from 
higher  education  and  real  practice  in  public  schools.  It  is  not  surprising  that  pre -service  teachers  trained  to  use  and 
integrate  technology  into  instruction  fall  into  this  gap  during  student  teaching. 

Literature  also  indicates  that  action  research  imparts  approaches  and  methods  that  serve  to  strengthen 
teacher-preparation  programs  (Price,  2001)  by  having  pre -service  teachers  share  their  field-based  research  findings 
to  a community  of  learners.  Rock  and  Levin  (2002)  have  documented  how  several  teacher  preparation  programs 
have  used  action  research  to  promote  skills  of  inquiry,  problem  solving,  reflection  and  critical  thinking.  The  idea  of 
conducting  action  research  in  teacher  preparation  programs  has  gained  support  (Keating,  Diaz-Greenberg,  Baldwin 
& Thousand,  1998)  because  it  becomes  a way  for  teachers  to  teach  other  teachers  (Feldman  an  Atkin,  1995)  in  a 
collaborative  mode  that  fosters  the  growth  of  learning  communities.  Recently  Pan  (1999)  has  argued  that  engaging 
teachers  in  action  research  is  also  an  effective  means  to  learn  to  integrate  technology  by  enhancing  communication 
and  collaboration  between  teachers  and  university  faculty.  He  supports  the  idea  that  if  teachers  understand  the 
significance  of  research  and  “conduct  action-research  of  teaching  with  computers’’  they  will  hold  positive  attitudes 
toward  learning  by  integrating  educational  technology  into  teaching.  Royer  (2002)  advocates  the  use  of  action 
research  as  an  “effective  professional  development  strategy’’  to  encourage  teachers  to  integrate  computer 
technology  into  their  teaching. 

Faculty  members  in  teacher  preparation  programs  across  the  country  also  show  increased  interest  in 
electronic  portfolios  as  a means  to  integrate  technology  and  for  authentic  and  performance-based  measures  (e.g., 
Bartlet,  2002;  McKinney,  1998).  As  noted  by  Wright  and  Stallworth  (2002),  with  the  development  of  National 
Educational  Technology  Standards  for  Students  and  Teachers,  the  use  of  electronic  portfolios  by  pre -service 
teachers  may  introduce  them  to  the  “skills  for  teaching  and  learning  in  future  classrooms’’.  Effective  electronic 
portfolios  lead  students  to  learn,  teach,  and  learn  to  teach  with  technology  as  they  reflect  on  the  artifacts  they  select 
from  their  teaching  experiences  (Curts  and  McWright,  2002).  There  is  evidence  that  electronic  portfolios  become  an 
important  instrument  in  the  on-going  efforts  to  prepare  tomorrow's  teachers  to  use  technology  critically  and 
reflectively. 

Implementation 

These  findings  supported  the  committee’s  conviction  that  the  task  of  ensuring  that  every  new  teacher  learns 
how  to  integrate  technology  into  teaching  and  learning  can  only  be  achieved  by  remo  ving  limitations  and  creating 
opportunities  for  teachers  to  experience  pedagogically  sound  application  of  instructional  technology  in  both 
university  classes  and  field  experience.  At  UTPA  there  is  a technology  class  for  the  elementary  teacher  preparation 
program  but  the  secondary  program  was  limited  by  the  fact  that  there  is  no  formalized  approach  to  include 
technology.  The  secondary  program  is  based  on  the  professional  development  school  model  and  includes  a field- 
based  component  that  puts  pre -service  teachers  in  area  classrooms  simultaneous  to  beginning  their  courses  in  the 
College  of  Education.  The  field  component  comprises  50%  of  student  activity.  The  secondary  courses  are  divided 
into  three  blocks  covering  three  semesters.  Courses  in  Block  I consis  t of  nine  credit  hours  that  include  a foundations 
course,  an  educational  theory  course  and  a content  area  reading  course.  Block  II  is  comprised  of  six  credit  hours 
that  encompass  methods  of  planning,  instructional  delivery,  management  and  assessment.  Block  III,  six  credit 
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hours,  is  the  final  student  internship  or  student  teaching.  The  secondary  program  is  limited  to  these  21  hours,  so 
there  was  no  option  to  add  a technology  course,  nor  did  research  findings  support  the  idea  of  a stand-alone 
technology  course.  The  need  to  restructure  the  curriculum  to  correspond  to  new  state  standards  was  evident.  The 
secondary  faculty  viewed  the  restructuring  process  as  an  opportunity  to  refine  the  existing  program.  Faculty 
members  were  particularly  interested  in  reports  of  how  action  research  methods  and  electronic  portfolios  could  serve 
as  a strategy  to  integrate  technology.  The  findings  of  the  Technology  Committee  and  these  considerations  assisted 
the  secondary  faculty  in  defining  the  following  goals  and  strategies  as  the  foundation  for  restructuring  the  secondary 
teacher  preparation  program. 


Goal 

1.  The  infusion  of  technology  into  teaching  and 
learning  will  be  the  norm  for  the  UTPA  secondary 
teacher  preparation  program. 


2.  Establish  internal  and  external  support  systems 
for  faculty  and  student  development. 


3.  Pre -service  teachers  need  more  opportunities  to 
see,  use  and  apply  IT  during  field  experience  and 
internships. 

4.  Action  research  processes  will  be  incorporated 
into  the  preparation  program  to  strengthen 
professional  development  and  as  a central  activity 
for  the  integration  of  technology  in  all  courses  and 
field  experience. 


Strategy 

la.  Align  courses  to  the  State  technology  standards 
for  all  teachers  so  that  the  necessary  skills  could  be 
distributed  throughout  the  curriculum  to  allow 
incremental  acquisition. 

lb.  Education  faculty  will  model  and  integrate 
technology  that  reflects  the  current  needs  in  both 
teacher  education  and  K-12  schools. 

lc.  Provide  education  specific  software,  not 
available  in  general  computing  labs,  for  student  use 
in  the  COE  labs. 

2a.  Collaborate  with  the  Center  for  Distance 
Learning  (CDL)  for  immediate  technical  support  to 
incorporate  web-based  activities  into  secondary 
courses. 

2b.  Seek  funding  through  a Preparing  Tomorrow’s 
Teachers  to  use  Technology  (PT3)  grant  application 
to  provide  additional  resources  for  faculty  and 
student  support. 

3a.  Seek  more  involvement  with  mentor  teachers  to 
achieve  integration  into  the  field  component. 

3b.  Establish  an  electronic  learning  community  to 
enhance  communication  with  mentor  and  intern 
teachers. 

4a.  Action  research  tasks  will  be  identified  and 
distributed  throughout  the  curriculum  to  result  in  a 
project  to  be  completed  during  student  teaching. 

4b.  Adopt  the  use  of  electronic  portfolios  as  part  of 
the  assessment  process  and  as  a vehicle  to 
demonstrate  artifacts  from  action  research  projects. 


The  first  and  crucial  activity  toward  implementing  these  strategies  was  identifying  technology  activities  and 
products  that  would  require  acquisition  of  the  skills  defined  in  the  State  technology  standards.  The  following  table 
lists  how  the  technology-based  activities  are  matched  to  the  standards  and  distributed  among  the  secondary  course 
Blocks. 
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Table  1 Distribution  of  technology  skills  within  secondary  courses 


State  Technology  Standards 

Standard  III:  Use  of  task  appropriate  tools 


Standard  IV:  Use  of  technology -based 
communication  and  productivity  tools 


Standard  V:  Use  technology  to  plan, 
organize,  deliver,  assess  and  evaluate 
instruction 


Technology  Applications  Course  Blocks 


I 

II 

III 

Word  Processing 

Mail  merge 

X 

X 

X 

X 

Tables,  charts,  graphs 

X 

X 

Spreadsheet  /Database 

X 

X 

Excel 

EasyGradePro 

Multimedia  (text,  audio,  graphics) 

X 

X 

PowerPoint,  Scanning,  ErontPage  (HTML) 
Digital  Video 
Desk-top  publishing 

X 

X 

X 

Draw  and  paint  applications 

X 

X 

Participate  in  electronic  communities 

X 

X 

Electronic  Learning  Community 

X 

X 

WebCT 

X 

X 

X 

Uploading/downloading.  FTP  protocols 

X 

X 

X 

Internet  search,  Web  Board  Discussion 

X 

X 

X 

Online  collaboration.  Email,  Chats 

X 

X 

X 

HTML  protocols 

X 

X 

Use  of  simulations 

X 

X 

Email 

X 

X 

X 

Electronic  Portfolio 

X 

X 

Concept  maps  on  PD  competencies 

X 

X 

Planning  aligned  to  the  TEKS 

X 

Online  quizzes  and  tests 

X 

X 

examples  of  simulated  student  reports 

X 

3 examples  of  assessment  instruments 

X 

Reflections 

X 

X 

X 

Philosophy  of  teaching  statement 

X 

Block  III  lesson  plans 

X 

Example  letters  to  parents  & students 

X 

Web-based  discussion 

X 

X 

X 

Bulletin  Brds,  Internet  Srch,  Online  Library  SRch 

X 

X 

X 

WebCT 

X 

X 

Inspiration  6.0 

X 

X 

X 

Internet  Search 

X 

X 

X 

PowerPoint 

X 

X 

X 

Frontpage 

X 

Online  Library  Search 

X 

X 

X 

Electronic  Learning  Community 

X 

X 

WebCT 

X 

X 

X 

Principles  of  Instmctional  Design 

X 

X 

X 

Learner  assessment 

X 

X 

X 

Content  assessment 

X 

X 

X 

Media  selection 

X 

X 

X 

Design  of  instmctional  treatment 

X 

X 

X 

Evaluation  of  process  & product 

X 

X 

The  secondary  faculty  felt  that  technology  skills  could  be  acquired  simultaneous  to  actual  integration 
activities  situated  in  teaching  contexts  rather  than  isolated  to  professional  development  activities.  The  technology 
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infusion  began  by  building  on  existing  UTPA  strengths  during  the  fall  semester  of  2000.  The  university  setting 
provides  a strong  centralized  technology  infrastructure  as  well  as  interest  in  the  infusion  of  technology  into  the 
teacher  preparation  program.  Prior  technology  initiatives  by  the  UTPA  Center  for  Distance  Learning  (CDL) 
contributed  a desirable  local  model  for  faculty  training  and  support  systems  that  effectively  linked  the  use  of 
technology  to  pedagogical  constructs.  The  CDL  had  established  that  technology  skill  and  implementation  could  be 
achieved  through  just-in-time  technical  assistance  and  an  ongoing  coaching  that  begins  by  assisting  faculty  to 
augment  their  traditional  courses  with  web-based  activities.  This  approach  fosters  incremental  skills  development 
specific  to  content  pedagogical  need  and  encompasses  most  of  the  web  and  Internet  skills  identified  in  the  State 
standards.  The  secondary  faculty  adopted  this  approach  and  coordinated  efforts  with  CDL  to  begin  augmenting  the 
Foundations  of  Education  courses  during  the  fall  semester  of  2000.  This  augmentation  introduced  faculty  and 
students  to  a wide  range  of  web-based  technologies  linked  appropriately  to  the  course  of  study.  Students  and  faculty 
alike  were  encouraged  by  the  enhanced  student  participation  achieved  in  web-based  discussion  and  the  convenience 
of  asynchronous  collaborative  activities.  Faculty  tracked  student  discussion  participation  in  traditional  classes  as 
compared  to  web-based  discussion  and  noted  that  the  average  5%  student  participation  rate  in  class  discussions  was 
increased  to  100%  by  simply  assigning  discussion  comments  to  be  posted  to  WebCT.  By  the  spring  semester  of 
2001  the  other  two  Block  I courses  were  augmented  with  WebCT  and  by  the  beginning  of  the  fall  semester  in  2001 
all  secondary  faculty  had  been  introduced  to  WebCT  and  used  it  in  their  courses  for  discussions,  collaborative  group 
projects  and  other  activities  during  spring  semester  of  2001.  This  augmentation  of  traditional  courses  with  distance 
education  procedures  served  as  a successful  strategy  to  infuse  the  use  of  technology  into  course  activities  as  a part  of 
instructional  procedure  and  changed  the  traditional  secondary  courses  to  “hybrid”  courses  that  successfully  infused 
web-based  communication  as  integral  part  of  instruction. 

With  the  award  of  a PT3  grant  in  the  summer  of  2001,  broader  support  systems  were  established  within  the 
College  of  Education  based  on  the  successful  approach  modeled  by  CDL  with  individualized  technical  support  and 
coaching.  An  instructional  developer  and  a technical  assistant  and  work-study  students  were  hired  to  expand  and 
continue  faculty  support  in  an  individualized  manner.  Initial  assistance  covered  a wide  range  of  activities  like  basic 
email,  creating  group  email  lists,  creating  personal  web  pages,  online  library  searches,  web  searches,  online  chats, 
online  quizzes,  use  of  PowerPoint,  spread  sheets,  uploading  and  downloading  documents,  file  transfer  protocols. 
Frontpage  and  Inspiration  concept  mapping  software.  This  individualized  approach  enabled  faculty  to  seek 
assistance  in  a manner  that  did  not  endanger  their  self-esteem  or  encumber  their  busy  schedules. 

At  that  time  surveys  were  administered  to  faculty,  pre  -service  teachers  and  mentors  to  establish  a profile  of  baseline 
competencies  for  faculty,  pre -service  teachers  and  high  school  mentors. 


Table  2:  Profile  of  computer  usage  among  faculty,  pre  -service  teachers  and  high  school  mentors 


Skills/Knowledge 

Secondary  Faculty  ‘‘ 

Secondary  pre -service 
teachers  ’’ 

High  School  Mentor 
Teachers 

General  use  of  computer 

100% 

80% 

30% 

General  use  of  a word 

100% 

90% 

79% 

processor 

Advance  features  of  a word 

80% 

N/a 

N/a 

processor 

General  use  of  spreadsheets 

100% 

50% 

84% 

Use  of  formulas,  functions 

90% 

50% 

21% 

and  graphs 

Create  report  (query /find)  in  a 

50% 

N/a 

N/a 

database 
Scan  documents 

90% 

70% 

N/a 

Email 

100% 

95% 

58% 

World  wide  web  to  retrieve 

100% 

95% 

66% 

information 

Online  database  search 

100% 

30% 

47% 

Online  library  services 

90% 

50% 

47% 

Teach/or  take  Online  course 

50% 

N/a 

N/a 

Multimedia  presentations 

50% 

70% 

26% 

Creation  of  Web  pages 

40% 

20% 

10% 
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“PT3-UTPA  Faculty  Technology  Use  survey  was  applied  on  Oct.  16,  2001.  Results  reported  here  correspond  only  to 
the  Secondary  Faculty  from  the  C & I Department.  The  summary  statistics  report  for  the  whole  sample  was  included 
in  the  PT3-UTPA  Interim  Activity  Report,  July  01,  200 1 - October  31,  2001. 

Results  reported  here  correspond  to  UTPA  incoming  pre -service  students  that  enrolled  into  the  Secondary  Teacher 
Preparation  Program  during  Fall  2001. 

A Mentor  Teacher  Pre -training  Technology  Survey  , developed  by  the  Center  for  Applied  Research  (CARE)  at 
UTPA,  was  applied  to  85  elementary,  47  middle  school  and  19  high  school  teachers  belonging  to  the  STCPDT 
collaborative.  The  data  above  relates  only  to  the  high  school  teachers. 

The  resulting  profile  for  UTPA  constituencies  corresponded  to  the  national  profile  indicated  by  research. 
Indeed,  individual  university  faculty  and  students  were  more  skilled  in  most  areas  than  mentor  teachers  but  had  not 
been  accustomed  to  incorporating  the  use  of  technology  into  instruction.  The  success  of  the  first  two  semesters  and 
the  addition  of  needed  staff  through  PT3  resources  provided  continued  support  for  faculty  and  student  activities  so 
attention  was  then  focused  on  mentor  teacher  involvement.  It  was  easily  recognized  that  the  simple  fact  of  physical 
separation  limited  opportunity  for  faculty  and  mentor  teacher  communication.  The  need  to  create  communication 
opportunities  between  faculty  and  mentor  teachers  was  addressed  in  the  PT3  Grant.  One  of  the  primary  objectives  of 
the  grant  was  to  create  an  infrastructure  for  an  electronic  learning  community  for  Rio  Grande  Valley  teachers.  The 
interest  in  an  online  environment  was: 

1.  to  ease  communication  and  consultation  among  university  faculty,  mentor  teachers  and  interns  separated  by 
distance; 

2.  to  collaborate  to  conduct  action  research  activities  that  models  skills  innate  to  reflective  practice; 

3.  to  affirm  the  essential  relationship  between  content,  theoretical  and  pedagogical  knowledge  as  the  basis  for 
reflective  practice; 

4.  to  support  development  of  technological  skill  and  understanding  of  technology -supported  learning; 

5.  to  encourage  technology-supported  instructional  activity  in  order  to  expand  possibilities  for  innovative 
instructional  solutions;  and, 

6.  to  demonstrate  a broader  process  for  professional  consultation  that  provides  example  of  collaboration 
empowered  by  web-based  communication. 

PT3  resources  facilitated  the  acquisition  of  the  already  existing  infrastructures,  the  Electronic  Learning  Community 
(EEC)  and  Electronic  Portfolio  (EP),  fromthe  Center  for  Technology  in  Education  (GTE)  at  Johns  Hopkins 
University  (JHU).  Acquiring  these  infrastructures,  instead  of  creating  them,  accelerated  not  only  the  grant  activities 
but  also  the  completion  of  the  program’s  stated  restructuring  goals  for  technology  infusion.  Arrangements  were 
made  with  Johns  Hopkins  GTE  staff  and  training  sessions  were  held  for  faculty  and  mentor  teachers  to  introduce 
them  to  the  use  of  the  EEC  and  EP.  Students  were  introduced  to  the  ELC  and  EP  as  part  of  the  courses  in  Block  I 
classes.  At  that  time,  Yanes  and  Curts  (2002)  proposed  an  online  communication  model  for  student  teaching  to 
promote  the  use  of  action  research  and  reflective  practice  and  provide  a vehicle  for  enhanced  involvement  of  the 
mentor  teachers.  The  model  is  implemented  in  coursework  and  through  the  ELC.  The  ELC  serves  as  a web-based 
communication  tool  to  bridge  the  physical  distance  between  university  faculty  and  field  placement  sites  and  enables 
frequent  communication  between  university  faculty  and  mentor-intern  teams  at  remote  sites.  In  this  model  the  6- 
step  cycle  of  action  research  activities  (Johnson,  2002)  serves  as  a cohesive  element  that  begins  within  university 
courses  and  culminates  during  the  final  internship  when  it  involves  the  mentor  teacher.  The  electronic  portfolio  is 
used  to  house  artifacts  produced  in  action  research  activities.  The  six  steps  of  action  research  are:  1)  define  a 
question;  2)  review  the  literature;  3)  plan  data  collection;  4)  collect  and  analyze  data;  5)  create  an  action  plan;  and, 

6)  share  the  findings  (Johnson,  2002).  These  steps  are  infused  into  the  secondary  preparation  program  in  the 
following  manner. 

• During  Block  I students  create  a philosophy  of  education  statement,  formulate  a research  question 
and  commence  a literature  review.  The  philosophy  activity  assists  students  to  identify,  synthesize 
and  personalize  theoretical  and  pedagogical  constructs  into  a clear  statement.  This  statement 
serves  as  a basis  to  identify  their  major  educational  interests  and  to  formulate  a question  for 
investigation  from  that  perspective.  The  literature  review  is  conducted  based  on  that  question. 

• During  Block  II  the  literature  review  continues  and  students  are  introduced  to  data  collection  and 
analysis  procedures. 

• During  Block  III,  the  student-teaching  internship,  students  cooperate  with  their  mentor  teacher  to 
collect  data,  and  to  design  and  enact  their  action  plan.  The  products  of  the  student  teacher’ s work 
are  placed  in  the  EP  and  the  student  teacher  and  mentor  then  report  the  results  of  their  project  to 
their  peers  using  the  ELC. 
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This  approach  introduces  preservice  teachers  and  mentors  to  a systematic  process  that  brings  research  to  the 
classroom  at  an  appropriate  level  and  establishes  a reflection  cycle  that  encourages  ongoing  evaluation  and 
improvement  of  instruction.  The  following  model  demonstrates  how  action  research,  use  of  the  electronic  learning 
community  and  electronic  portfolio  moved  to  a central  position  in  the  secondary  teacher  preparation  program  and 
created  a focus  for  technology  infusion.  In  effect  the  ELC,  the  EP,  and  action  research  provided  the  mechanism  and 
the  impetus  for  ongoing  communication  and  collaboration  among  student  teachers,  mentors  and  university  faculty. 

EIGURE  1.  Action  research  technology  integration  model. 


The  end  product  for  each  student  is  the  final  report  of  the  completed  action  research  project  posted  to  the  ELC.  The 
EP  houses  all  action  research  activities  pursued  during  the  three-semester  program  as  well  as  the  final  report.  Since 
the  final  action  research  project  report  constitutes  30%  of  the  final  grade  for  student  teaching,  mentors  become  more 
involved  in  the  student  interns’  success.  This  higher  level  of  involvement  stimulates  the  mentors’  interest  in  the  new 
technology  methods  and  approaches  that  the  students  demonstrate  in  the  electronic  portfolio  and  include  in  the 
action  research  treatment. 

Conclusion 

To  date,  the  goals  stated  by  the  secondary  faculty  have  been  achieved  and  the  strategies  continue  to  prove 
effective.  The  technology  skills  acquired  through  use  of  distance  education  procedures  and  other  activities 
distributed  throughout  the  curriculum  allow  faculty  and  students  to  develop  technology  skill  incrementally  while 
attaining  their  immediate  educational  goals.  Although  the  PT3  Grant  now  provides  in-house  technical  support,  the 
department  continues  to  collaborate  closely  with  the  CDL.  The  creation  of  hybrid  courses,  combining  traditional 
and  distance  education  methods,  prepares  students  and  faculty  to  successfully  use  the  ELC  as  the  primary  means  of 
communication  during  the  final  student  teaching  internship.  One  unexpected  outcome  of  the  collaboration  with 
CDL  is  the  creation  of  an  online  program  for  the  secondary  professional  development  courses.  Bringing  the 
mentors  into  the  action  research  project  and  the  ELC  during  student  teaching  introduces  them  to  the  convenience  of 
technology -assisted  communication  and  encourages  them  to  communicate  with  university  faculty  and  other 
members  of  the  ELC  regarding  their  intern’s  action  research  project.  It  also  appears  to  motivate  the  mentors’ 
continued  activity  in  the  ELC  after  the  internship.  University  faculty  also  continue  to  collaborate  closely  with  the 
Johns  Hopkins  Center  for  Technology  in  Education  to  evaluate  ELC  usage,  expand  implementation  strategies,  and 
conduct  research  regarding  the  use  of  the  ELC.  This  model  seems  to  provide  students  and  university  faculty  with  a 
shared  focus  for  technology  infusion  activities.  It  also  appears  to  regenerate  mentor  teachers’  interest  in  the  infusion 
of  technology  into  the  classroom  teaching  and  learning  cycle.  Action  research  has  become  the  common  goal  for 
faculty,  students  and  mentors.  It  is  an  activity  that  contributes  to  strengthening  the  professional  development  of 
teachers  and  encourages  intellectual  and  pedagogical  growth.  The  ELC  collaborative  activities  in  use  at  UTPA  seem 
to  naturally  foster  the  growth  of  a learning  community.  In  this  context,  action  research  can  be  viewed  as  a 
constructivist  process  in  which  the  ELC  provides  a social  situation  where  teacher’s  beliefs  about  learning,  their 
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students  and  their  conceptions  of  themselves  as  learners  are  explicitly  examined,  shared,  challenged,  supported  and 
even  supervised.  These  electronic  “co-laboratories”  provide  avenues  for  geographically  dispersed  teachers  to 
exchange  ideas,  information,  best  practices,  and  offer  support  to  each  other. 
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Abstract 

One  of  the  most  recent  technological  revolutions  is  Internet,  which  provides  an  opportunity  for  teachers  and 
students  to  explore  information  and  to  learn  at  school,  at  home,  or  in  the  community.  Since  vast  amount  of 
information  is  available  on  Internet,  it  will  be  an  inevitable  resource  for  teachers  and  students  in  developing 
countries  as  in  developed  countries.  In  this  study,  gender  differences  in  middle  school  students’  attitudes  toward 
educational  uses  of  Internet  and  the  Internet  usage  in  Turkey  were  investigated. 

Introduction 

The  Internet  is  an  international  network  of  computer  networks  that  allows  its  users  to  share  information  and 
to  communicate  interactively  (Gunderson  & Anderson,  1999).  Acceptance  of  the  Internet  as  a potential  educational 
tool  has  provided  rich  teaching  and  learning  environments  for  students  and  teachers.  For  example,  teachers  have 
used  the  Internet  as  a communication  tool  (Wells  & Anderson,  1995),  as  an  information  seeking  environment  (Wei 
He  & Jacobson,  1996),  for  gathering  content  information  (Sunal,  Smith,  Sunal,  & Britt,  1998),  for  delivering 
instruction  (Sunal  et  al.,  1998),  for  linking  students  to  their  peers  worldwide  (Gersh,  2001),  and  for  posting  student 
work  on  the  Web  (Branzburg,  2001). 

Research  that  has  been  done  for  college  students  indicated  that  students  who  keep  track  of  valuable 
educational  sites  on  the  Internet,  share  information  with  their  friends,  and  use  the  Internet  very  often  for  educational 
and  general  reasons,  had  more  favorable  attitudes  toward  educational  use  of  the  Internet  (Duggan,  Hess,  Morgan, 
Kim,  & Wilson,  1999).  Experience  with  using  the  Internet  can  vary  for  college  students.  Wei  He  & Jacobson  (1996) 
indicated  that  social  science  undergraduates  had  less  Internet  experience  than  those  in  other  programs.  However,  all 
of  the  participants  in  this  study  thought  that  Internet  was  very  useful  tool  in  education.  The  general  attitude  of  the 
undergraduate  students  reflected  that  the  use  of  Internet  was  for  seeking  information  and  not  for  entertainment. 
Research  carried  out  for  8th  to  10th  grade  students  indicated  that  student  attitudes  toward  computers  became 
significantly  less  positive  during  their  junior  high  school  careers.  However,  it  was  argued  that  the  use  of  the  Internet 
to  access  very  exciting  ideas  and  rich  resources  and  materials  were  attractive  characteristics  of  the  Internet  in  order 
to  increase  positive  attitudes  toward  the  use  of  the  Internet  for  educational  purposes.  With  the  help  of  the  Internet, 
students’  attitudes  towards  the  use  of  computers  in  their  classes  could  also  be  increased  (McKinnon,  Patrick  Nolan, 

& Sinclair,  2000). 

Since  the  availability  of  the  computers  and  the  Internet  access  in  primary,  middle  and  high  schools  has 
increased  most  recently,  the  research  studies  in  grades  1 to  12  are  not  as  many  as  in  college  and  university  level. 
Studies  investigating  the  gender  issues  indicated  that  male  students  have  more  positive  attitudes  toward  use  of 
computer  and  high  self  confidence  in  use  of  technology  than  females  (Brosnan,  1998;  Chua,  Chen,  & Wong,  1999; 
Shashaani,  1997).  Study  of  Nachmias,  Mioduser,  & Shemla  (2000)  in  grades  7 to  12  revealed  that  there  was  a 
significant  difference  between  males  and  females  in  terms  of  time  spent  on  the  Internet,  preferred  location  for  use, 
resources  downloading,  Web  site  creation,  and  participation  in  discussion  groups.  Similar  to  previous  researchers 
Nachmias,  Mioduser,  & Shemla  (2000)  also  argued  that  extensive  use  of  the  Internet  in  schools  and  communities 
will  eliminate  acceptance  of  the  common  idea  of  the  Internet  usage  is  a type  of  males’  activity. 

A review  of  the  literature  reveals  that  in  Turkey,  similar  to  other  countries,  there  is  little  information  about 
middle  school  male  and  female  students’  attitudes  towards  the  use  of  the  Internet  and  computers  as  an  educational 
tool  in  their  courses.  There  are  a nurrber  of  reasons  why  so  little  is  known  about  students’  attitudes  towards  use  of 
technology  in  their  classes.  Technological  support,  especially  for  costly  technologies  such  as  computers  and  Internet 
accessibility,  is  not  provided  equally  for  all  schools  due  to  economical  constraints  in  Turkey.  In  general,  students  are 
familiar  with  the  use  of  video,  overhead  projector,  television,  microscopes,  and  etc.  as  part  of  instructional  materials 
in  their  courses.  Since  very  few  of  the  schools  have  computer  laboratories,  the  use  of  computers  in  courses  as  an 
instructional  tool  is  not  a common  teaching  strategy  in  regular  classrooms,  especially  in  public  schools.  Students  are 
normally  allowed  to  use  computers  for  their  computer  classes,  projects  or  to  seek  information  in  their  free  times. 
Yildir  (2001)  found  that  access  to  computers  in  schools  was  the  most  important  parameter  in  having  positive  attitude 
towards  computers.  In  addition,  it  is  indicated  that  gender  and  age  also  significantly  affected  the  elementary  school 
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students’  attitudes  towards  computers.  Clearly,  further  and  more  extensive  studies  are  needed  to  evaluate  the  middle 
school  students’  attitudes  towards  the  educational  use  of  the  Internet  in  Turkey. 

Recently  the  Internet  use  has  experienced  a dramatic  growth  in  Turkey  (Wolcott  & Cagiltay,  2001).  Most 
of  the  students  access  to  the  Internet  at  out  of  school  places,  instead  of  their  schools.  For  example,  they  are  using  the 
Internet  at  the  Internet  cafes,  at  their  homes,  or  from  their  friends’  computers.  In  parallel  with  the  huge  number  of 
resources  available  outside  the  schools,  it  is  clear  that  students  have  attitudes  towards  potential  educational  use  of 
the  Internet. 

The  purpose  of  this  study  was  to  present  the  results  of  attitudinal  data  from  Turkish  students  with  respect  to 
the  educational  uses  of  the  Internet  and  the  Internet  usage.  Main  focus  will  be  on  gender  differences. 

Methodology 

Participants  and  Data  Collection 

This  study  reported  data  from  a sample  of  Turkish  students.  All  of  the  students  were  enrolled  in  the  sixth, 
seventh,  and  eighth  grades  of  middle  schools  and  were  taught  by  the  same  curriculum.  All  schools  were  public 
schools.  The  schools  are  located  in  rural,  urban,  and  suburban  areas  near  and  in  Ankara,  Turkey.  For  this  study,  6 
schools  were  selected  randomly  among  the  public  schools.  For  each  grade,  one  classroom  was  selected  randomly  in 
each  school.  Data  were  collected  from  708  students  in  18  classrooms.  The  age  range  of  the  students  was  10  to  14. 

Instruments 

The  Attitude  Toward  Educational  Uses  of  the  Internet  (ATEUI)  scale  developed  by  Duggan,  Hess,  Morgan, 
Kim,  & Wilson,  (1999)  was  used  in  this  study.  The  ATEUI  scale  constituted  18  items  on  Likert  format.  Since  our 
participants  were  middle  schools  students,  we  modified  some  of  the  items  according  to  level  of  our  participants. 
Since  students’  access  to  the  Internet  was  expected  to  vary  with  regard  to  their  school  characteristics  and  students 
out  of  school  opportunities,  we  had  to  determine  those  parameters  to  make  sound  interpretation  about  the  findings. 
Eor  this  reason,  we  used  the  Behavioral  Correlates  Questionnaire  (BCQ)  developed  by  Duggan  et  al.  (1994).  This 
questionnaire  included  11  items. 

Results 

Descriptive  Analysis  of  Male  and  Female  Students’  Responses  to  BCQ  Items 

Table  1 summarizes  the  percentage  of  male  and  female  students’  responses  to  BCQ  items.  The  general  use 
of  the  Internet  increased  during  last  two  years  for  males  and  females.  Eor  example,  only  one  girl  indicated  that  she 
has  been  using  the  Internet  for  more  than  4 years  while  4.1%  of  the  males  indicated  so.  Similarly  3.3%  of  males  and 
1.2%  of  females  said  that  they  have  been  using  the  Internet  for  3-4  years.  However,  19%  of  males  and  16%  of 
females  said  that  they  were  using  the  Internet  for  less  than  a year.  This  data  revealed  that  the  Internet  has  become 
more  popular  for  both  males  and  females  most  recently. 

Among  the  students  23.3%  of  males  and  14.2%  of  females  indicated  that  they  learnt  to  use  the  Internet  by 
themselves  and  18.2%  of  males  and  9.1%  of  females  indicated  that  they  learnt  it  from  their  friends.  Magazines, 
books,  school,  and  library  were  not  preferred  as  a way  to  learn  the  use  of  the  Internet. 

The  students  (34.5%)  prefer  to  connect  to  the  Internet  from  their  homes  and  Internet  cafes.  Even  though 
nearly  equal  percentage  of  males  (14.6%)  and  females  (9.4%)  connect  to  the  Internet  from  their  homes,  the  analysis 
also  revealed  that  more  males  (26.3%)  prefer  to  use  Internet  cafes  than  females  (8.8%). 

Among  different  features  of  the  Internet,  males  mostly  preferred  to  use  the  chat  (27.1%),  WWW  (24.1%), 
e-mail  (14.6%),  download  (7.6%),  newsgroups  (6%),  and  ICQ  (5.4%).  Eemales  also  stated  similar  usage  patterns 
except  for  newsgroups:  chat  (16.2%),  WWW  (13.9%),  e-mail  (9.4%),  download  (4.7%),  and  ICQ  (3.5%).  This  data 
revealed  that  both  males  and  females  are  using  the  Internet  for  communication  and  search. 


Table  1.  Descriptive  Analysis  and  Chi  Square  Analysis  for  Male  and  Eemale  Students’  Responses  to  BCQ  items 


Gender 

Male 

Eemale 

Behavioral  Correlates 

N % 

N 

% Chi  Square 

How  long  have  you  been  using  the  Internet? 
Less  than  one  year 

71  19.2 

57 

16.8  35.45** 
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1-2  years 

32 

8.7 

15 

4.4 

35.45** 

2-3  years 

24 

6.5 

7 

2.1 

35.45** 

3-4  years 

12 

3.3 

4 

1.2 

35.45** 

More  than  4 years 

15 

4.1 

I 

0.3 

35.45** 

How  did  you  learn  to  use  the  Internet? 

Myself 

86 

23.3 

48 

14.2 

9.63** 

Books 

9 

2.4 

5 

1.5 

0.85 

Magazines 

18 

4.9 

6 

1.8 

5.21* 

Eriends 

67 

18.2 

31 

9.1 

12.03** 

School 

5 

1.4 

7 

2.1 

0.53 

Library 

2 

0.5 

I 

.03 

.026 

Where  do  you  connect  to  the  Internet? 

Home 

55 

14.9 

40 

II.8 

1.47 

School 

6 

1.6 

2 

0.6 

1.69 

Internet  cafe 

97 

26.3 

30 

8.8 

36.50** 

Eriend 

26 

7 

15 

4.4 

2.23 

What  of  the  following  features  for  using  the  Internet  apply  to 
you? 


E-mail 

54 

14.6 

32 

9.4 

4.47* 

WWW 

89 

24.1 

47 

13.9 

11.97** 

E-mail  list 

40 

10.8 

22 

6.5 

4.20* 

Etp 

8 

2.2 

1 

0.3 

4.94* 

Newsgroups 

22 

6 

5 

1.5 

9.70** 

Download 

28 

7.6 

16 

4.7 

2.50 

Telnet 

6 

1.6 

2 

0.6 

1.70 

Chat 

100 

27.1 

55 

16.2 

12.22** 

ICQ 

20 

5.4 

12 

3.5 

1.45 

Which  of  the  educational  purposes  do  you  use  the  Internet  for? 

Homework 

56 

15.2 

37 

10.9 

2.81 

Term  paper 

87 

23.6 

71 

20.9 

0.70 

Consult  with  instructor 

10 

2.7 

1 

0.3 

6.74* 

Consult  with  classmates 

24 

6.5 

10 

2.9 

4.88 

Retrieving  class  notes 

61 

16.5 

45 

13.3 

1.47 

Eollow  educational  sites 

33 

8.9 

21 

6.2 

1.89 

**  p < .01,  * p < .05 


Even  though  66%  (465)  of  the  students  indicated  that  they  did  not  connect  to  the  Internet  at  all,  the  analysis 
revealed  that  87%  (616)  of  the  students  used  the  Internet  for  educational  purposes.  The  students  mostly  used  the 
Internet  for  preparing  term  papers  (males  23.6%,  females  20.9%),  doing  homework  (males  15.2%,  females  10.9%), 
following  educational  sites  (males  8.9%,  females  6.2%),  and  doing  other  educational  activities  (males  15%,  females 
7%).  These  findings  suggest  that  students  give  importance  to  get  information  from  the  Internet  for  their  courses. 

Females’  and  Males’  Attitudes 

The  t-test  analysis  (t  =-.657,  p>.05)  suggested  that  both  females  and  males  have  favorable  attitudes  toward 
the  educational  uses  of  the  Internet. 

Chi  Square  Analysis 

Male  and  female  students’  responses  to  behavioral  correlates  were  analyzed  by  using  chi  square  test  to  find 
out  differences  between  their  Internet  literacy.  Table  1 summarizes  results  of  all  of  the  chi  square  analysis.  Analysis 
revealed  that  male  students  had  significantly  more  experience  in  using  the  Internet  than  female  students.  Even 
though  there  is  still  significant  difference  between  male  and  female  students’  number  in  using  the  Internet  during 
last  two  years,  a great  deal  of  increase  in  number  of  female  students  was  easily  observed. 
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Myself,  magazines,  and  friends  were  found  as  favorite  sources  for  both  male  and  female  students  to  learn 
how  to  use  the  Internet.  However,  when  we  compare  the  number  of  the  students,  significantly  more  male  students 
used  these  three  sources  to  get  information  about  the  Internet  usage. 

Both  male  and  female  students  were  connected  the  Internet  from  their  houses,  friends,  and  s chools. 
However,  significantly  more  male  students  preferred  to  use  Internet  Cafes  to  connect  to  the  Internet. 

Among  the  different  features  of  the  Internet,  e-mail,  www,  e-mail  list,  ftp,  news  groups,  chat  were  used 
significantly  more  by  male  students  than  female  students. 

Using  the  Internet  for  educational  purposes  was  similar  for  both  male  and  female  students.  Only  significant 
difference  was  obtained  for  consulting  instructor.  More  male  students  preferred  to  use  the  Internet  to  communicate 
with  instructor  through  the  Internet  than  female  students. 

Discussion 

Descriptive  analysis  and  t-test  analysis  revealed  that  during  last  two  years  both  males  and  females  have  had 
an  interest  in  using  the  Internet  for  different  purposes.  Our  results  and  other  research  results  show  that  gender  did 
not  create  differences  in  attitudes  toward  educational  uses  of  the  Internet  (Duggan  et  al.,  1994).  Since  most  recently 
both  females  and  males  are  spending  nearly  equal  time  on  the  Internet  (Internet  users,  2001;  Jackson,  Ervin,  and 
Garden,  2001),  they  have  favorable  attitudes  toward  the  Internet. 

Even  though  both  male  and  female  students  had  generally  positive  attitudes  toward  educational  uses  of  the 
Internet,  significant  differences  were  obtained  in  terms  of  their  Internet  literacy.  Similar  to  Nachimas  et  al.,  (2000) 
findings,  this  study  supported  that  more  male  students  used  the  Internet  than  females  for  different  purposes. 
Moreover,  significant  differences  were  obtained  between  genders  regarding  with  age  of  starting  to  use  the  Internet, 
sources  to  learn  the  Internet,  places  to  connect  to  the  Internet,  activities  done  by  using  the  Internet,  and  obtaining  the 
information  and  communicating  with  teachers  and  friends  for  different  educational  purposes. 

Interestingly,  our  data  suggested  that  even  though  only  33.6%  of  the  students  indicated  that  they  were 
actively  using  the  Internet,  87%  of  the  students  indicated  that  they  were  using  the  Internet  for  educational  purposes. 
This  finding  revealed  that  even  though  students  had  very  little  opportunity  to  access  the  Internet  through  their  school 
resources,  they  found  different  ways  (such  as  Internet  cafes,  friends  computers  etc.)  to  access  the  information 
through  Internet  for  their  homework  and  term  papers.  It  is  clear  that  students  already  involved  in  using  the  Internet 
for  educational  purposes  as  a reliable  source.  Teachers  also  supported  their  students  to  use  the  Internet  for  their 
homework,  term  papers,  and  other  educational  activities.  Otherwise  students  would  not  use  the  Internet.  This  result 
strongly  suggests  that  school  administrators  should  provide  more  opportunities  for  students  to  easily  access  the 
Internet  in  their  schools  to  increase  the  effectiveness  of  teaching  and  learning.  Earlier  studies  suggest  that  instruction 
effectiveness  can  be  increased  by  using  the  Internet  (Berge,  1997;  Eollansbee,  1997).  This  high  demand  to  use  the 
Internet  for  educational  purposes  might  rapidly  increase  the  Internet  access  at  schools. 

Another  finding  of  this  study  revealed  that  communication  through  chat,  e-mail,  newsgroups  and  web 
browsing  were  other  reasons  for  using  the  Internet  for  middle  school  students.  This  result  indicated  that  students  also 
use  the  Internet  for  entertainment  purpose.  This  finding  supports  the  Nachimas  et  al.,  (2000)  study. 

This  study  will  add  to  the  literature  and  also  will  help  those  actively  involved  in  educational  uses  of  the 
Internet  research  and  those  participating  in  the  improvement  of  education  in  developing  countries  like  Turkey.  This 
study  suggests  that  s tudents  are  using  the  Internet  to  obtain  information  for  their  courses.  School  administrators  and 
teachers  should  be  informed  about  the  ways  of  their  students’  using  the  Internet.  In  this  way  they  can  provide  more 
opportunities  to  the  students  according  to  their  interests  and  increase  the  effectiveness  of  the  instruction  and 
students’  success. 
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